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i. 0 j. 0 ' D-a d-c 

(13) 

We will represent the matrix of z 4tJ as the sequence of 
input approximation coefficients {c 0 } . Process 68 
concludes by storing the sequence {c 0 } in image memory 67. 

It has been found that the sequence (S 1#J f ) (x,y) has 
important properties that are useful in the compression 
of documents according to the preferred embodiment of the 
invention, and particularly in the compression of two- 
color, or binary, documents. Firstly, the sequence 
(S r<J f) (x,y) is a bilinear spline function that is 
continuous over the rectangle [a, b] by [c, d] , and its 
restrictions on the subdivided sub- rectangles U it x M ) by 
tyj' yj.xJ-for integers i from 0 to 2 X , and integers j from 
0 to 2 J are all bilinear polynomials. Secondly, the 
sequence (S 1(J f) (x,y) agrees exactly with the sampled 
input signal sequence f <x lf y,) at the grid points. 
Thirdly, the sequence (s x>J f) (x,y) has the second order of 
approximation, in that: 



\(S Z J f) (x,y) -£{x,y) \z (constant) 2-*<™ 

(14) 

for all points (x,y) within the rectangle [a, b] by [c, 
d] , and for all smooth functions f(x,y), with the 
constant being independent of the number of grid points 
and grid lines. 

Following projection process 68, spline-wavelet 
decomposition may now be done, in the general sense, by 
decomposition of the input approximation coefficient 
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sequence {c°} into low-pass and band-pass components. The 
abHaty to perforin such decomposition is due to the 
multi-resolution analysis properties discussed above. 

For purposes of explanation, in a one -dimensional 
case, let one -dimensional functions f i (x) and g } (x) be 
represented as follows: 



k 

(14) 
(15) 

From multi-resolution analysis, an approximated signal 
f„(x) can be decomposed into the sequence: 

U) (x) +...-g w (*) ♦/„_„,(*) 

, (16) 
where m' < m, and where g, (x) e W m and f i (x) s v,. 



Recalling from multi -resolution analysis that spl 
space v 3#l = v, e Wj , that v s c V w and that w, c w, lf 
terms of the functions f(x) and g(x), decomposition may 
be considered as: 



me 

in 



t )tl (X) = f d { X ) + g 3 (x) (17) 

In this embodiment of the invention, the function g, (x) 
provides the higher- frequency details of the image (i e 
edges and corners) at the level of decomposition, 
while the function f i (x) provides a low-frequency 
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approximation of the image. While one -dimensional 
functions f (x) and g(x) are used hereinabove for 
explanation, it is of course to be understood that two- 
dimensional decomposition functions f(x,y) and g(x,y), as 
useful in the decomposition of a two-dimensional 
functions representative of a document, may be obtained 
in a similar way (i.e., via the tensor product). 

One can thus determine a relationship between a 
higher-level approximation coefficient sequence {c^ 1 } and 
its lower- level component approximation coefficient 
sequences {c 3 }, {d 1 } as follows: 

(18) 
(19) 

indicating that spline -wavelet decomposition according to 
the present embodiment of the invention may be performed 
by convolution of the approximation coefficient sequences 
with decomposition sequences {a n }, {b„} . 

Accordingly, referring now to the illustration of 
the generalized method of Figure 7 for the decomposition 
of a document, subprocesses 70 and 71 of decomposition 48 
are performed upon the input signal coefficient sequence 
{c 0 }; Figure 7 shows subprocesses 70, 71 operating upon a 
generalized sequence {c J }, considering that the input to 
processes 70 and 71 will be a previously decomposed 
component sequence in the case where decision 51 (Figure 
2) requires additional decomposition to obtain the 
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TZFJZZT* ratio - According to this — 

invention, processes 70, 71 analyze the incut 
sequence in row-wise fashion fir«r c I 
MrfnriB(! "smon first. Subprocess 70 

performs the convolution of the sequence (cM with a 
coefficient sequence /a.) tv, af ' 
select i • M corresponds with the 

selected linear B- spline fnn^, „« 
r6la . . spune runction, as noted above 

relative to Figure 5a Th,-» , 

followed bv a !T convolution of process 7 o is 

this desampling may be an inherent result of the 
convolution operation. The result of process 72 Is 
sequence {c>-}, which is then stored H u 
memory 67. red back lnto ima 9e 

Similarly, to derive the row-wise high-pass 
component of the input sianai , • 

ocyuence (c n ) that corresponds wit->, 

function m process 71 ™t " , Sele " ed WaVelet 
is fouled bv „V "-volution of process 71 

operation is discarded f- ? convolution 

sample. The resuit L ' p OC e s^ .'ST 
which *.u Process 73 is the sequence (d^ 1 ) 

which „ then stored back into image memory 6 7 along w 4 h 
the low-pass sequence {e *-) i„ process 72. * 

decor~t::-ri:u": ^r^r in w r the 

decoded, but in a coW Jl e LnL ™ 

subprocess 73 convolves the blur image i e W ■ , 
both spline function coefficient sequence la ) J , 

function coefficient sequence ,b„,. ToUoled^t ^ 
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incorporating therein a downsampling by a factor of two. 
The result of subprocess 73 is the blur portion of the 
two-dimensional image (referred to as the LL component) 
corresponding to the column-wise low-pass filtered 
5 component of row-wise low-pass filtered result {c^ 1 }, and 
a higher- frequency component (referred to as the LH 
component) corresponding to the column-wise low-pass 
filtered component of the row-wise high-pass filtered 
result . Similarly, subprocess 74 convolves and 

10 downsamples the wavelet component {d** 1 } with both spline 
and wavelet function coefficient sequences {an}, {b n }, 
respectively. The result of subprocess 74 is the high- 
frequency component of the two-dimensional image referred 
to as the HL component, which corresponds to the column - 

15 wise high-pass filtered component of the row-wise low- 
pass filtered result {c J - 1 }, and the highest frequency 
component referred to as the HH component, which 
corresponds to the column-wise high-pass filtered 
component of the row-wise high-pass result {d j ' x } . The 

20 component results LL, LH, HL, HH are stored back into 
image memory 67 in process 75, for quantization by 
process 50 of the flow of Figure 2. 

(i) Dual Base Wavelet Decomposition 

25 

According to a first embodiment of the invention, 
decomposition process 48a takes advantage of the duality 
principle for certain wavelets, as described in Chui, et 
al., "Fast integral wavelet transform on a dense set of 

30 time-scale domain", CAT Report 310 (Center for 

Approximation Theory, Texas A&M University, September 
1993) . Through use of the duality principle, it has been 
discovered that each element of both decomposition 
coefficient sequences {a n }, {b n } , and of both 

35 reconstruction coefficient sequences {p n }, {q„} may be 

expressed as a rational number. Integer scaling of these 
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sequences thus allows for all of the matrix operations of 
process 48a to be integer operations, and can thus be 
performed very quickly and efficiently by integer-based 

60 ' SUCh as the ™S 320C25 or the like, in spline- 
wavelet compressor 20. it is therefore preferred 
according to the present invention, that the default 
condition of spline -wavelet compressor be set to perform 
decomposition according to process 48a, using the dual 
wavelet approach. 

For purposes of explanation, and as noted in the 
above-referenced Chui et al. paper, a "dual" function 
* B (x) of the mth order B-spline function * B (x) exists, 
such that: 



and such that: 



$„U) e clos Li span I ^(x-k) | kez ) 

(20) 



*' J 0 k*l 



(21) 



It is preferred that the order m be even -numbered; the 
most useful case is the linear case (m=2), in which: 



(22) 



Because of this relationship, one may perform wavelet 
decomposition in dual space, using linear dual spline 
function 0 2 (x) together with linear dual wavelet f unct 
*a<x) (i.e., the dual of linear wavelet function tf a ( x ) ) 



ion 
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Linear wavelet function ^ 2 (x), in this example, is 
preferably the following function: 



i|r 2 (x) : =£ qjs 2 (2x-n) 

a 

(23) 

where, for m generally: 




w=0 ,1, . . 2m~2 



(24) 

For a discussion of these functions in the general case 
(for order m) , attention is directed to Chui, An 
Introduction to Wawi^p (Academic Press, 1992) , 
incorporated herein by this reference; particular 
attention is directed to pages 182-186 in this regard. 
By way of the duality principle noted above, the dual ot 
linear wavelet function ^ (x) may therefore be expressed 
as follows: 

* . (25) 

Id. , at 185. 

By way of further explanation, use of the dual base 
wavelet decomposition technique will first be described 
relative to a one-dimensional sampled input signal 
f(k/2 J ), where 2 J is the number of sample points in the 
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input sequence and where k is the sample point index in 
the sequence. Projection of the sampled input signal 
onto spline space results in the function f j(x ) as 
follows: 



(26) 



The function f a (x) thus provides a continuous 
representation of the sampled input signal f (k/2') As 
noted above relative to process 68 of Figure 7, since the 
approximation coefficient sequence {c k '} is exactly the 
or lgi nal sequence of inpuf signal samples, projection of 
the function £,( X J into spline space as a set of 
approximation coefficients requires no computation in 
thas case and results in the original sampled input 
signal sequence f (k/2 J ) . 

Next, using the duality principle according to this 
embodiment of the invention, the continuous function 
t,(x>, as represented by the approximation coefficient 
sequence represented directly by the sampled input signal 
sequence f <k/2'> , is changed into a dual approximation 
coefficxent sequence by way of a coefficient sequence q* 
determined by equation (22) above as follows: 



6 2 J 2 J 2 J 



(27) 



Referring back to the decomposition tree of Figure 
5a, the following dual space decomposition formulae may 
be readily derived: 
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l-2k-m 

(28) 

£ 

J-2Jr-3m*2 

(29) 

using the finite decomposition sequences a k/ b k , as 
follows : 



if — 0 f 1 , • , . , jt? 



(30) 



m f > 



£=0, . . . ,3in-2 



(31) 



In the linear (m=2) case, the sequences a k( b k equal: 



{a Q ,a l ,a 2 )=l{i,2 t l), a k =0 for all other k ' 

(32) 

ib 0 ,b lt b 2 ,b 3 ,b 4 }'-^{i t -6, 10,-6,1), b k =0 for all other k 

(33) 

Further, the reconstruction sequences {p k }, {a*} equal the 
decomposition sequences {a k , b k } . m addition, reference 
to the summation limits in equations (28), (29) will show 
that the convolution operation will also result in 
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desampling of the results, by a factor of two. As will 
be Seated hereinbelow. the results of this convolution 
operation win provide both low- pas, and high-pass 
cogent results that can be stored in the sen* memory 
-pace as that which originally contained the input 
setjuence . 



As is evident from the foregoing, the decomposition 
sequences (a k , h k} will be ^ advantageQus ^ fche 10n 

standpoint of implementation into modem data processing 

equipment, since each sequence is a fin^o « 

yuence is a finite sequence of 

rational numbers. The decomposition routine is thus very 
Similar to a digital m (finite impulse response) 

48a 11' dUal ^ WaVel " process 

48a xs very efficient when performed by spline-wavelet 
compressor 20 on two-dimensional input data 
representative o, a document, as will now be described 
relative to Figures 8 and 9. 

Decompression process 48a of Figure 8 begins with 
process 76 , in which the two-dimensional input signal 
coefficient sequence (c'J is projected into dual saline 

ZTi^^T^' ^ 3 in the --nclature 

coe«i" T" 0rd " ° £ d «™"°". su=i> that the 
coetficient sequence (C) refers to the original input 

signal coefficient sequence, m the two-dimensionT 
case, input signal coefficient sequence {c ') refers to the 
sequence ( S „„ (x , y) . Process ?6 generates ^ 

ITZITS™ ^ " 3 SerlM ° f "cording 
to the following sequence, analogously to equation (22, 
above, as follows: 

a i.J=<Ci- l , J - 1 *c J . 1 , J . 1 .c J . 1 . ji *c 1 . l . J . 1 ) n6 c J ^4(c J . 1 . J »c i . J . 1 »c J . J .,» 
where 1341 
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c -i,j = c z*.i.j = c i.-i = c i. 2 '*i = 0 for a* 1 i'J 



As a result, one may express the relationship between the 
spline space coefficient sequence {c j } and its dual 
5 coefficient sequence {c 3 } as follows: 

(35) 

Upon completion of process 76, the dual coefficient 
10 sequence {c j } is stored back into image memory 67. 

Process 78 and 80 are then performed by way of which 
the row-wise decomposition processes are performed by DSP 
60 using integer operations, as enabled by the dual base 

15 wavelet technique according to this embodiment of the 
invention. As noted above, the input sequence {c 3 } 
consisted of a two-dimensional array of values in PGM 
format; as such, the dual sequence {c 3 } similarly consists 
of such an array. Process 78 performs convolution of 

20 each row of the dual array sequence (c j } with a sequence 
{a*} derived as noted above, in the manner of a low-pass 
filter. In this example, where order m - 2 (i.e., linear 
spline-wavelet analysis) , the convolution of process 78 
consists of the point -wise multiplication of each row in 

25 dual sequence {c j } with a sequence {a k } derived according 
to equation (2 8) noted above: 
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(28) 



where Jc is the position of each entry within the row of 
dual array sequence As noted in equation (32) for 

the linear case, the sequence {a k } is defined as: 

la 0t a 1 .a 2 Ul (1,2,1), 3jt =o for all other k 



(32) 



The 1/4 term may be applied to the resultant data later 
allowing the convolution of process 78 to be performed ' 
solely with integer values. As noted above, the 
convolution of process 78 results in a convolution result 
{e» l ) that is inherently downsampled by a factor of two. 
For example, only the even-numbered values of the 
convolution results may be retained (given the summation 
limits noted above) , with the odd-numbered values 
discarded. As will be evident hereinbelow. this 
downsampling allows for the same image memory space to be 
used to maintain the decomposed image. 

Similarly, process 80 performs convolution of the 
dual space array sequence {c>} with a sequence {b k } , 
derived as noted above, in the manner of a high-pass 
filter. As in the case of process 78, where the order m 
=2 (i.e., linear spline -wavelet analysis), the 
convolution of process 80 consists of the point-wise 
multiplication of each row in dual sequence {&) with a 
sequence {bj according to equation (29) noted above: 
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5 r x = E 

(29) 

As noted in equation (33) above, for the linear case, the 
sequence {bj may be calculated as follows: 

{b Q ,b lt b 2 ,b 2 ,b A )=-±{l,-6,10,-6,l}, b k =0 for all other k 

(33) 

Again, the 1/24 division may be performed at a later 
time, to allow the convolution of process 80 to be 
performed using only integer values. As noted above, the 
row- wise convolution of process 80 provides results {d 5 * 1 } 
that are downsampled by a factor of two, e.g. by 
retaining only the even-numbered entries. 

Upon completion of the row-wise decomposition and 
downsampling of processes 78, 80, the downsampled 
decomposed sequences {c^ 1 }, {d*" 1 } are stored in image 
memory 67, in a form in which the left-hand side of the 
image corresponds to the low frequency results {c^ 1 }, and 
in which the right-hand side corresponds to the high 
frequency results {d j - 1 } . Attention is directed to Figure 
9 for a visual representation of the operations of 
processes 78, 80. Array 90 indicates the contents of 
input dual array sequence {c j }, prior to row- wise 
decomposition. The inherent downsampling in the results 
of each of processes 78, 80 provides a combined 
downsampled sequences {c^ 1 }, {d 3 * 1 } that is stored by 
process 82 in image memory 67 in the same memory space as 
the original array, as shown by array 92 of Figure 9. 
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The results of the row-wise 

decomposition are then subjected to column-wise 
decomposition, for both low-pass and high-pass cases, in 
processes 84, 86 respectively. Process 84 retrieves each 
column of the stored results for sequence { e M} (from 
array 92), and performs a decomposition into another pair 
of low and high frequency sequences in the same manner as 
process 78 discussed above, except that the index k now 
refers to positions within the particular column under 
analysis. The results of this decomposition of process 
84 are inherently downsampled by a factor of two (e g 
by keeping only even-numbered values), as discussed' " 
above . 



Similarly, process 86 performs low-pass and high- 
pass decomposition of the results on a column-by- 
column basis. The decomposition of process 86 is a 
convolution in similar f ashiori as discussed hereinabove 
except that the index * now refers to position within the 
column under analysis. The results of column- wise high- 
pass decomposition process 85 are also inherently 
downsampled by a factor of two, as noted above. 

The results of processes 84, 86 then correspond to 

st°or r e rrr nts of the input duai «* «■ 

stored back xnto image memory 67 in process 88. These 
components are referred to as the LL, LH, HL, HH 
components. By way of explanation, the LL component , 
consists of those sequence entries resulting from column- 
wise low-pass process 84 upon the row-wise low-pass 
results of process 78; the HL component consists of 

the results of column-wise low-pass process 84 upon the 
row-wxse high-pass results {d^} of process 80 
Sxmilarly, the LH component consists of the results of 
column-wise high-pass process 86 upon the row-wise low- 
pass results (el") of process 78, while the HH component 
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consists of the results of column-wise high-pass process 
86 upon the row-wise high pass results {d 5 * 1 } of process 
80, Figure 9 illustrates, with array 94 , the arrangement 
of these four components in image memory 67, with A=LL., 
B-HL, C=LH, and D=HH . 

Upon storing of the decomposed components back into 
image memory 67 in process 88, the input document image 
is ready for quantizing in process 50, as will be 
described hereinbelow. It should be apparent to those of 
ordinary skill in the art, particularly with reference to 
equations (32) , (33) , that this embodiment of the 
invention provides the significant advantage that spline- 
wavelet decomposition may be performed using spline and 
wavelet functions of finite support, and which involve 
only integer calculations. As such, the decomposition 
process of this embodiment of the invention may be done 
at very high speed, similar to a digital FIR filter, or 
alternatively with relatively primitive processing units, 
while maintaining high quality of the compressed 
document , 

In addition, it will become apparent, from the 
description hereinbelow regarding decompression of the 
document, that reconstruction of the document may be made 
using the same spline and wavelet sequences {a k }, {b k } as 
used in decomposition of the document. As such, 
reconstruction may also be performed solely with integer 
operations, and thus at high speed or with inexpensive 
processors . 
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lijj Interpol atory wawl fl » 

Referring back to Figure 2, according to an 
alternative embodiment of the invention, decomposition 
process 48 may be performed by way of interpolatory 
wavelets (shown as process 48b) . As noted above, it is 
contemplated that interpolatory wavelet decomposition 
will be most useful where the compression and 
decompression speed is of the highest importance, at a 
cost of somewhat reduced reproductive quality. 

According to this embodiment of the invention, the 
document image is decomposed by way of the centralized B- 
spline scaling function, together with its corresponding 
wavelet. Figure 10a illustrates the centralized linear 
B-spline function * L (x), which is defined as follows: 



<M*>= { 1_W 

0 o Chez wise 



(36) 



This expression is a unit left shift of the cardinal B- 
spline of equation (1) above. The corresponding wavelet 
function * L (x) to centralized linear B-spline function 
4>u(x) is as follows: 



■ <l> t (2x-i) 

(37) 



and is illustrated in Figure 10b. The wavelet function 
iMx) has an interpolatory property, and as such is 
commonly referred to as an interpolatory, or 
interpolating, wavelet. Further information regarding 
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interpolatory wavelets may be found in Chui and Li, 
"Dyadic Affine Decompositions and Functional Wavelet 
Transforms", CAT Report (Center for Approximation 

Theory, Texas A&M University, April 1993) . 

From the linear B-spline, scaling function ^(x) may 
be expressed as follows: 

<M*). = -|<|> t (2x+l) ♦ $ L (2x) * !<t> t (2x-l) 

(38) 

then : 

$ L (2x) = $ L (x) - ±y L (x+l) - -|t L (x) 

(39) 

Using the general wavelet decomposition formula of: 
4>(2x-ic) - 5>*-aj*U-.Z> + Vb k . 21 y(x-1) 

(40) 

one may then readily show that, for the centralized 
linear B-spline scaling function ^(x) and its 
interpolatory wavelet \p L (x) : 

{a„} ={..., 0, l, o, . . .) 

{*>»} - {• • •/ 0, l, 0, . . .} (41) 

both over non-negative integers. 

Process 48b of Figure 2 is thus performed 
identically as process 48a, described above relative to 
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Figure 8, except that the decomposition sequences {a„}, 
{b n } of equations (41) are used in processes 78, 80, 84 
86. As such, the interpolator wavelets of this 
embodiment of the invention provide the same advantages 
in decomposition, as described hereinabove relative to 
the dual base wavelet technique of process 48a, in that 
integer operations may be utilized for such 
decomposition . 

However, differently from the dual base technique 
described hereinabove, the use of interpolator wavelets 
require the reconstruction sequences { Pk} , {qk ) to be 
different from decomposition sequences {aj, {b n } . From 
equation (39) above: 



(42) 



As such, the reconstruction sequence {pj f or the 
interpolator wavelets (the use of which will be 
described in detail hereinbelow) will have p , = « p - 
1, and Pl = - Hf witn all othfir valueg of p eguai ^ 

Given the simple expression of equation (37) for ^(x) 
the reconstruction sequence {q,} will have only a single 
non-zero element, namely * . x , with all other q£ 
q equal to zero. 



.wavelet 

decomposition using S niino. u , V6lp , p a „i^. 

According to another alternative embodiment of the 
invention, specifically the third approach noted above 
decomposition process 48 may be performed according to 
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another technique which will be referred to herein as 
"wavelet packets" or "spline wavelet packets". As will 
be apparent from the description hereinbelow, this 
decomposition (or compression) approach utilizes a full- 
tree decomposition, rather than the half-tree approach 
discussed above. Full- tree decomposition refers to 
decomposition in which both the low-pass and the high- 
pass (or band-pass) components are decomposed when going 
to the next level of decomposition. 

The use of orthogonal wavelet packets in wavelet 
analysis is known, as indicated by Coifman, Meyer, et 
al . , "Signal Processing and Compression with Wave 
Packets" , (1992). However, these wavelet packets are not 
spline -based, and are not symmetric, and as such their 
use in fields such as image compression and decompression 
would introduce undesirable phase distortion. 

According to this alternative embodiment of the 
invention, process 48c (Figure 2) performs decomposition 
of the document image by way of non- orthogonal, 
symmetric, spline-based wavelet packets. The 
mathematical theory behind this class of wavelets is 
presented in Chui and Li, "Nonorthogonal Wavelet 
Packets", SIAM J. Math. Anal . . Vol. 24, No. 3, (SIAM, May 
1993), pp. 712-738. According to the present invention, 
these spline -based wavelet packets are used to further 
decompose the results of data, corresponding to a 
document, that are decomposed to a first level by 
application of a scaling function and the corresponding 
wavelet function. This further decomposition provides 
higher compression ratios and higher quality picture 
reproduction, when compared against the dual -base 
technique (process 48a of Figure 2) described 
hereinabove . 
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In contrast to the decomposition approach 
illustrated in Figures 5a, 5b, where only a single branch 
(the {c n i} branch of Figure 5a) is applied to subsequent 
decomposition stages, both of the low-pass and high-pass 
components are further decomposed according to this 
wavelet packet technique. 

Figure lla illustrates the decomposition "tree" 
according to this alternative embodiment of the 
invention. As is evident therefrom, two-dimensional 
nomenclature is required to indicate the decomposition 
component of interest, since components of the n+1 level 
can come from either low-pass or high-pass results from 
the prior level; the nomenclature used in the tree of 
Figure lla identifies each component sequence {c"-"} as 
the m" lowest frequency component above the spline 
component in the n th level of decomposition. For example, 
sequence {c 2 - 3 ) is the sequence of coefficients 
representative of the second lowest frequency component 
above spline component {c 0 - 3 } in the third level of 
decomposition. 

Figure lla illustrates that each component sequence 
{c-- n } is decomposed, in the next level of decomposition, 
into a low-pass and a high-pass component, each of which 
is itself decomposed in the following level of 
decomposition. In the tree of Figure lla, the input 
coefficient sequence {c 0 - 0 } (corresponding to the input 
signal function f (x) ) is convolved with scaling function 
coefficient sequence {a n } to produce a low-pass component 
coefficient sequence {c 0 - 1 }, and is also convolved with 
wavelet function coefficient sequence {b„} to produce a 
high-pass component coefficient sequence {c 11 } . For the 
next level of decomposition, low-pass component 
coefficient sequence {c 0 - 1 } is convolved with scaling 
function coefficient sequence {aj to produce a second 
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level low-pass component coefficient sequence {c 0 - 2 }, and 
is also convolved with wavelet function coefficient 
sequence {b n j to produce a high-pass component coefficient 
sequence {c 1 - 2 } . To this point, the decomposition is 
similar to that described hereinabove relative to the 
dual-base decomposition and interpolatory wavelet 
decomposition . 

In the next level, however, according to the wavelet 
packet technique according to this embodiment of the 
invention, the first level high-pass component 
coefficient sequence {c 11 }, which is basically a wavelet 
component, is also convolved with wavelet function 
coefficient sequence {bj to produce a low-pass component 
coefficient sequence {c* 2 }, and is also convolved with 
scaling function coefficient sequence {a n } to produce a 
high-pass component coefficient sequence {c 3 ' 2 }. One 
should note that the coefficient sequences applied to a 
previous high-pass component are reversed from that of 
the tree of Figure 4a, such that the high-pass component 
of a previous high-pass component is obtained by 
convolution with. scaling function coefficient sequence 
{a n }, and so that the' low-pass component of a previous 
high-pass component is obtained by convolution with 
wavelet function coefficient sequence {b„} . Additional 
levels of decomposition can then continue, for each of 
the component coefficient sequences {c"- n } so generated, 
until the desired compression ratio is obtained. 

The basic spline scaling function and corresponding 
wavelet function utilized according to this alternative 
method of the invention is governed by a family of basis 
functions. This family includes a series of functions n lt 
where n 0 is a spline scaling function and where each 
function ji 4 {for i > 0) is a wavelet function. In the 
general case, where /i 0 (x) is equal to the scaling function 
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0(x), the first wavelet function ^ will equal the 
corresponding wavelet function *( X ) , and the progression 
of wavelet functions will be as follows: 



and (43) 

(44) 

As a result, the wavelet packet basis functions (i can be 
generated iteratively, in the general sense, by using the 
scaling function *<x) and the corresponding wavelet 
function <Hx). Accordingly, a component coefficient 
sequence {c B . n } at any level of decomposition, including 
the original input coefficient sequence {c"'«} can be 
obtained by the linear combination of wavelet packets. 

By way of explanation, decomposition according to 
this alternative technique will now be described in the 
case of a one-dimensional function f (x) . First, as in 
the prior cases, the input function f (x) is projected 
into the scaling function, or spline, space as follows: 

fix) = £c;-°u 0 (2^-j) =£c/'°4.(2x-j) 

J 3 

(45) 

The wavelet packet decomposition of {c 0 - 0 } is then 
governed by the following formulae: 
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k 

(46) 

where : 

^ = j 0 for even k 
* 1 for odd k 

(47) 

and where: 

/ *n . / {a J for A * = 0 
{2n ' 1 (bj for A, = 1 

(48) 

The coefficient sequences {aj, {b n } are, again, the 
spline and wavelet coefficient sequences determined hy 
selection of the scaling function and wavelet function 
selected for the decomposition. According to this 
embodiment of the invention, the linear and cubic 
coefficient sequences {a n }, {b n } preferred for use in the 
wavelet packet decomposition of digital documents are 
listed in Appendix A hereto. Decomposition is then 
performed according to the tree diagram of Figure lla, to 
the extent desired. 

As will be described in further detail hereinbelow, 
reconstruction of a decomposed representation is 
performed by convolution of high-pass components with a 
reconstruction coefficient sequence {qj and low-pass 
components with a reconstruction coefficient sequence 
{Pk}/ in full-tree form, as shown in Figure lib. The 
reconstruction formula utilized in this operation is as 

follows : 
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(49) 



Referring now to Figures 12a, 12b, 13, and 14 
process 48c in which a two-dimensional function f (x y) 
representative of a document, expressed as an array'of 
sample values in byte-packed PGM format, is decomposed 
according to the wavelet packet technique, will now be 

trilTJ" detai1 ' ShOWn in F±gUre 12a < in P~— 
96, DSP 60 projects the input data into spline space (as 

« coefficient sequence {c ---}, . As described hereinabove 
for a digital array of data as provided by format 
converter 12, the coefficient sequence corresponds to the 
values m the digital array f (x , y) . The arrangement of 
input coefficient sequence {c 0 ,} in an array of 
coefficients (x) is shown in Figure 13a, which is at the 
beginning of the decomposition tree of Figure 14 . 

Decomposition of the input coefficient sequence 
{Co.o) (or array {x}) then begins with processes 98, 100 
in which the sequence {<=„} is convolved, in row . wise ' 
fashion, with decomposition coefficient sequences (a } 
{b n } to produce the low-pass and high-pass component 
coefficient arrays {x}, { A} , respectively. Decomposition 
coefficient sequences {a a} . {b n} are based upon scaling 
functions and wavelet functions suitable for use in this 
wavelet packet approach, such as those shown in Appendix 
A. Each convolution in the decomposition sequence 
contains downsampling by a factor of two, as in the cases 
discussed hereinabove. As such, as shown in Figure 13b 
each row of the image data contains a row-wise low-pass' 
component coefficient array {x} and a row-wise high-pass 
component coefficient array {a} . For purposes of 
description of Figures 13 and 14, the low-pass component 
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coefficient sequence will retain the designation of the 
sequence from which it was generated (hence array {X} in 
Figure 13b is the low-pass component of array {x} in 
preceding Figure 13a) . 

The result of process 98, namely the low-pass 
component of the input image data array is then again 
decomposed, in row-wise fashion, in processes 102, 103. 
Process 102 convolves coefficient array {X} of Figure 13b 
with spline decomposition coefficient sequence {a„}, 
including downsampling, to generate another row-wise low- 
pass component, shown as coefficient array {X} in Figure 
13c. Process 103 convolves coefficient array {X} of 
Figure 13b with wavelet decomposition coefficient 
sequence {b n }, including downsampling, to generate a row- 
wise high-pass component, shown as coefficient array {B} 
in Figure 13c. 

The high-pass results from process 100 are similarly 
decomposed again, in a row-wise direction, in processes 
104, 105. Process 104 convolves the high-pass 
coefficient array {A} of Figure 13b with wavelet 
decomposition coefficient sequence {b„}, including 
downsampling, to produce a low-pass coefficient array {A} 
of Figure 13c. Similarly, process 105 convolves high- 
pass coefficient array {A} of Figure 13b with spline 
decomposition coefficient sequence {a„} , including 
downsampling. to produce a high-pass coefficient array 
{C} of Figure 13c. As noted hereinabove, according to 
this wavelet packet technique, the spline decomposition 
coefficient sequence {a„} and wavelet decomposition 
coefficient sequence {b n } are reversed, in this second 
decomposition pass when applied to the high-pass results 
of the first decomposition pass. 
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The results of processes 102, 103, 104, 105 
(together corresponding to the arrangement illustrated in 
Figure 13c) are then each applied to column-wise 
decomposition processes 106LL, 106LH, 106HL, 106HH, 
respectively. Figure 12b generically illustrates the 
subprocess steps performed in each of column-wise 
decomposition processes 106LL, ioslh, 106HL, 106HH. 

Referring to Figure 12b, subprocess 108 performs the 
low-pass decomposition of its corresponding components 
(see Figure 13c) in a column-by-column manner, by 
convolving the coefficient arrays with spline 
decomposition coefficient sequence {a n } in the manner 
discussed above, which includes downsampling by a factor 
of two. As a result, each of the coefficient arrays {x} , 
{B}, {A}, {C} of Figure 13c provide low-pass coefficient 
arrays {X}, {b}, {a}, {c} , respectively, as shown in 
Figure 13d. Subprocess 110 similarly convolves its 
corresponding coefficient array of Figure 13c with 
wavelet decomposition coefficient sequence {b n } and 
downsamples the result by a factor of two. As a result, 
each of the coefficient arrays {X}, {B] , {a}, {c} of 
Figure 13c thus also provide high-pass coefficient arrays 
{E}, {f}, {g}, respectively, as shown in Figure 13d. 

The results of this first column-by- column 
decomposition step are then repeated for a second pass, 
as before. The low-pass results of subprocess 108 are 
convolved in a corresponding subprocess ill, in column- 
wise fashion, with spline coefficient sequence {a n j , 
including downsampling; the cumulative effect of 
subprocess 111 for all of the results of processes 102, 
103, 104, 105 is to produce low-pass coefficient arrays 
(*) i {B}, {A}, {C} of Figure 13d. Subprocess 112 
convolves its corresponding high-pass coefficient array 
W. {B}, {A}, {C} of Figure 13d, in column-wise fashion, 
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with wavelet coefficient sequence {b„}, with the results 
downsampled, to cumulatively produce high-pass 
coefficient arrays {h}, {j} , {l}, {n} , respectively. 
Similarly, each of the high-pass coefficient arrays {D}, 
{E}. {F}/ {G} of Figure 13d from subprocess 110 are 
convolved, in a corresponding subprocess 113, in column- 
wise fashion, with wavelet decomposition coefficient 
sequence {b D j (downsampled) to cumulatively produce low- 
pass component coefficient arrays {D}, {e}, {f}, {G} 
shown in Figure 13e. Each of high-pass coefficient 
arrays {D}, {E}, {F}, {G} of Figure 13d are also 
convolved in a corresponding subprocess 114, in column- 
wise fashion, with spline decomposition coefficient 
sequence {a„}, including downsampling, to cumulatively 
produce high-pass component coefficient arrays {i}, {k}, 
{M}, {0} of Figure 13e. 

The results of subprocesses ill, 112, 113, 114 in 
each of processes 106LL, 106LH, 106HL, 106HH are then 
stored by process 116 in image memory 67, in the form 
illustrated in Figure 13d. Accordingly, the full-tree 
decomposition of Figure 14 is complete at this time, with 
the "blur" image of the input image sequence represented 
by the lowest -pass component of the row-wise and column- 
wise decomposition, represented by coefficient array {X} 
of Figures I3e and 14. All of the other coefficient 
arrays {a} through {0} of Figure 13 e represent wavelet 
packet components from this decomposition. 

According to this alternative embodiment of the 
invention, the decomposition tree of Figure 13e, and in 
general the decomposition tree of Figure 11a, may be 
"pruned", or stopped, at any particular component. 
Referring to Figure lla, it may be possible to stop the 
decomposition process for an intermediate decomposition 
component, for example at sequence {c 2,2 },- in this 
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example, the reminder of the coefficient eequencee in 
SL^T "° Uld te 0btalMd by ™»"on ^ the 

(="^ouid ; except that neither of se9uences - 

(c } would be generated. This aUows for a reduction in 
the amount of data to be atored. and proceased in 
aubsequent levels, particularly if no additional 
compression thereof is available. For example a 
particular sequence result of a decomposition may have 
all values equal to one another; further decomposition 
would not provide any additional compression 

pro^LT? ° V " thaC " hiCh 3 iOSSleaS -uld 
provide. Pruning of the decomposition tree at such a 

result would thus be advisable. 

When pruning of the full decomposition tree 
according to the wavelet packet technique is used, 
however, the arrangement of data in the resulting array 
must also be indicated within the data stream sent tT 
downstream processing units (i.e.. decompression system 
30) . m order to communicate which components of the full 

Figure 15. an example of the data frame useful in 
communicating the results of wavelet t>act»h x. ■ ■ 

will now be described Th, „ I . decomposition 
aescrioea. The data frame shown in Figure is 
corresponds to the format aa atored in image memo^ 67 
upon complete of the decomposition process (prior to 
the quantization, lossless compression, and other 
processes described hereinbelow) , and may also correspond 
to the compressed image data as communicated over network 
26 or as stored on disk 22, 24 (after quantization, 
lossless compression, and the other processes described 
hereinbelow) . 

As shown in Figure 15, data frame lie consists 
primarily of a data portion 120, which contains the 
coefficient sequence results of the decomposition process 
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described above. Header 119 indicates the beginning of 
data frame 118, and contains conventional information 
regarding the length and identity of data frame 118, as 
does header portion 123; trailer 121 follows the data 
5 portion 120 to indicate the end of data frame 118, and 
may also contain identity information. Part of the 
overall header of data frame 118 (i.e., the information 
preceding the actual data portion 120) according to this 
embodiment of the invention is tree map 122, which 
10 contains information concerning the components contained 
within data portion 120, especially considering the 
availability of "pruning" of the decomposition tree 
allowed according to the full-tree wavelet packet 
decomposition method described hereinabove. 

15 

Tree map 122 includes a first word 124 that 
indicates the number of decomposition levels performed. 
In this embodiment of the invention, map words 126 are 
provided for each decomposition level to indicate which 

20 components are present at each level of decomposition; as 
such, word 124 is necessary to indicate the number of map 
words 126 to follow. Each map word 126 contains a number 
of characters {which may be single or plural bits) equal 
to the number of possible components for that level; in 

25 other words, the m ch level of decomposition has 2™ 

possible decomposition components. As such, map word 12€ 0 
has one character as it corresponds to the zeroeth level 
of decomposition; it may therefore be possible, given the 
construction of header 119, 123, to eliminate map word 

3 0 126 0 since it merely indicates the presence or absence of 

any data in data portion 120. First level map word 126 x 
contains two characters, considering that the first level 
decomposition results in one low-pass component and one 
high-pass component. Second level map word 126 2 contains 

35 four (2 2 ) characters, third level map word 126 3 contains 
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eight (2 3 ) characters, and so on, for the number of 
decomposition levels performed. 

Figure 16 illustrates the decomposition tree of 
Figure 14, pruned so that the sequence {A} of Figure 13c 
was not further decomposed, and that sequences {B} and 
{C} of Figure 13d were not further decomposed. If, for 
example, each character in each of map words 126 contains 
a "i» to indicate the presence of a component and 
contains a "0" to indicate the absence of a component 
because of pruning, the following table indicates the 
contents of map words 126 (left -most bit containing the 
character for the lowest frequency component) : 

126 0 = ! 
126 x = ii 
126 2 = mi 
12 *3 - 1111 0011 

126, = mi 0011 0000 0011 

As a result of this construction of data frame 118, a map 
of the component sequences obtained by the full -tree 
decomposition 48c may be communicated along with the 
sequences. Of course, other techniques for communicating 
this information may alternatively be used. 

As in the case of the previous examples of this 
embodiment of the invention, the operations required of 
DSP 60 in performing the wavelet packet decomposition of 
process 48c are integer operations, given the integer 
nature of the decomposition coefficient sequences {an}, 
{b n } of Appendix A hereto. As such, the complex 
operations of process 48c may still be performed 
relatively quickly, and by processors of moderate 
capability; of course, the wavelet packet decomposition 
(and reconstruction, as will be described hereinbelow) 



zeroeth level 
first level 
second level 
third level 
fourth level 
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requires additional computing resources compared to the 
dual base technique and the interpolatory wavelet 
techniques described hereinabove. 

The wavelet packet technique provides a high degree 
of compression (by a factor of 16, for example) in each 
pass, as compared to the dual base technique and the 
interpolatory wavelet techniques. In addition, because 
of the full tree decomposition, the quality of the 
reproduction from the compressed data will be somewhat 
higher than the other techniques. As such, the wavelet 
packet technique described herein will be attractive for 
certain high quality applications, and as such is a 
useful option within compressor system 20 according to 
this embodiment of the invention. 



iii^ Completion of t he compression operation 

Regardless of the particular decomposition option 
selected as process 48 of Figure 2, the results of the 
decomposition operation, as stored in image memory 67, 
are then ready for the remainder of the compression 
process, as will now be described relative to Figure 2. 

Referring back to Figure 2, process 50 is next 
performed by quantization processor 62 upon the results 
of the decomposition stored in image memory 67. 
Specifically, the wavelet, or high-frequency, components 
of the spline-wavelet decomposition process 48 (i.e., the 
LH, HL and HH components of the dual -base wavelet 
decomposition and the interpolatory wavelet 
decomposition, and the wavelet packet components of the 
wavelet packet decomposition) are subjected to 
thresholding and quantization in process 50. It is 
preferred that quantization is not performed upon the 
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"blur" components (i.e., ll, or lx\) .u- 

^ . ' or l A /'/ as this component 

contains most of the significant information of t £ e ° ent 

Z^lT' ■° d ,"" SUCh ^ — thresholding 

would cause loss of quality without a significant 
increase in the compression ratio. 

However, quantization of the high-frequency wavelet 
components of the decomposition may readi^be IZTo^l 
since most real-world images in PCM format will consist 
primarily of low-frequency intensities, such that the 
higher-frequency wavelet components will tend to have a 
large number o, smaU, or zero, coefficient values 

TZltZ \° h Che i Praferred — " «- invention, 
hi!h I POE,Ul " lon «* —11 values in the 

discarded reqUenCy ^" — l-"*- may are 

discarded as a result n-F . 

f the quantization of process 50 
The memory requirements for storage of the coeff ic en ts 
that undergo the thresholding and quantization are thus 
*ucn reduced, even before the application of lossless 
compression techniques as will be noted below. 

As noted above, quantization processor 62 is a looic 
circuit for filtering the data corresponding to 
decomposed images in order to achieve the desired 
compression ratio, based upon one of a number of 
™ *" n "*«ion — ». The simpiest technique for 
performing quantization is referred to as "thresholding" 
where any coefficient having an absolute value less til' 
a certain threshoid value is set to zero, and where aU 
remaining coefficient values are rounded to the nearest 
integer value. The other quantization modes indicated 
above may instead be performed in process 50 by 
quantization processor 62, according to the known 
conventional techniques noted above. 
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Upon completion of the quantization of process 50, 
decision 51 is performed to determine if the desired 
compression ratio has yet been achieved. Upon the 
completion of a single decomposition process 48, the blur 
5 component of the image may be adequate to accurately 

convey the input image. This single decomposition of the 
image will provide up to a 4:1 compression ratio (for the 
dual-base and interpolatory wavelet case) or 16:1 ratio 
(for the wavelet packet case) , depending upon the memory 

10 requirements for the quantized high frequency results. 

This maximum ratio is obtained if all higher frequency 
components are discarded in the quantization process 50, 
leaving only the blur component. Decision 51 thus 
determines if the compression ratio achieved so far is 

15 adequate for the desired transmission or storage and, if 
not, passes control back to the decomposition process 48 
so that the blur component from the prior decomposition 
may again be decomposed according to the selected 
optional technique. 

20 

According to this embodiment of the invention, the 
determination of whether the desired compression ratio 
has been obtained may be done relative to a predetermined 
compression ratio. In such a case, data controller 56 
25 will maintain a count of the number of passes through 

decomposition process 48, and will perform decision 51 by 
comparing the resulting compression ratio against a 
previously stored value. 

30 Alternatively, the compression ratio decision 51 may 

be determined in a dynamic manner according to the 
accuracy with which the blur component is representative 
of the input frame image. Conceptually, such a 
determination will be a measure of the difference between 

35 the blur component and the input image block relative to 

a predetermined accuracy limit, such that if the blur 
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decomposition is within a preselected e l imit , „ 
additional pass through the decomposition process 48 may 
be performed, it is contemplated that this determination 
can be made automatically, by data controller 56 in 
decompressor 20 so that the process performed in each of 
channel compression subsystems may remain consistent, for 
example by calculating a numerical value based upon the 
sum of the coefficients in the high-frequency components 
of the decomposed image, which will indicate the 
difference between the input image to the blur image 
result of the decomposition process 48. 

whether statically or dynamically determined, upon 
decision 51 returning the result that the desired 
compression ratio has been obtained, according to the 
preferred embodiment of the invention, lossless 
compression is then performed upon the results of the 
decomposed and quantized images for the frame, in process 
52 ot Figure 2. Referring back to Figure 4, the lossless 
compression of process 52 is preferably performed by 
lossless compressor 64 according to a conventional 
lossless technique such as Huffman encoding. The 
lossless compression of process 52 is especially 
beneficial in compressing the quantized higher frequency 
components from the decomposition, considering that non- 
zero or varying values in these components will be quite 
sparse : for most video image frames. 

After the lossless compression of process 52, the 
compressed image data is then formatted for transmission 
or storage, as the case may be. it is preferred that the 
coding of the compressed image data be performed within 
compressor 20, preferably by data flow interface 66 shown 
in Figure 4, prior to its application to digital 
communications network 26 or bus 25. Besides the actual 
decomposition sequences, as compressed, the transmitted 
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compressed document image data must include information 
regarding the type of compression performed; such 
information must identify the type of lossless 
compression (if any) , the spline-wavelet decomposition 
5 process, and the type of byte-packing and number of color 
bits, used in the compression process. It is 
contemplated that such coding, including the insertion of 
this necessary data control information and document 
identity information as a header or trailer. Examples of 

10 such information include x-dimension image size, y- 

dimension image size, the number of bits used in byte 
packing, the wavelet decomposition method used (i.e., 
dual base wavelet, interpolatory wavelet, wavelet 
packet) , the number of levels of decomposition, the 

15 quantization method used, and the lossless compression 
method used. This and equivalent information and 
arrangements will be within the knowledge of one of 
ordinary skill in the art having reference to this 
specification. 

20 

3 . The decompression system 

The construction and operation of decompression 
25 system 30 according to the preferred embodiment of the 

invention will now be generally described. As in the 
case of compression system 10, decompression system 30 
may be a stand-alone system, but is preferably arranged 
so as to be an add-on card for a conventional high 
30 performance personal computer or workstation or as a 
function of a larger computer, such as a mainframe 
computer or supercomputer. It is also contemplated, as 
will be apparent from the following description, that it 
will be possible to implement decompression system 30 
35 with the same hardware as compression system 10, 

considering that decompression system 30 is performing 
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substantia!^ the inverse operation, of those ascribed 
hereinabove, and as such uses o.ite similar circuitr ^ 0 
accomplish the same. 

However, it is also contemplated that many 
applications will utilize a central compression system 
10, with many remote decompression systems 30 able to 
receive or retrieve compressed image data therefrom, m 
such an arrangement, there is no need for the remote 
stations to have compression capability, as they will be 

read-only" systems. Accordingly, the construction and 
operation of decompression system 10, and specifically 
spline-wavelet decompressor 40, as a decompress -only 
subsystem will now be provided. 

As discussed above and as shown in Figure 1 
decompression system 30 includes spline- wavelet ' 
decompressor 40 and format converter 32. Spline-wavelet 
decompressor 40 receives the compressed image data from 

26 ' ° r frora disk borage 22, 24 via bus 27, and 
reconstructs the document image from the decomposition 
sequences in the manner to be described hereinbelow 
Format converter 32 operates to reformat the decompressed 
document into the suitable form for display on video 
display 34d, printing by printer 34p, or output by way of 
another conventional output device. 

Referring now to Figures 17 and 18, the construction 
and operation of spline-wavelet decompressor 40 to 
reconstruct, or decompress, the received or retrieved 
document image data according to the preferred embodiment 
of the invention will now be described in detail 
Spline-wavelet decompressor 40 includes data controller 
156 which controls the operation of the other components 
within spline-wavelet decompressor 40. m this example 
data controller 156 presents and receives signals on ' 
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control bus 163 to control the timing, feedback and 
transmission of information through and from spline- 
wavelet decompressor 40. It is therefore contemplated 
that data controller 156 may be implemented as a 
5 relatively simple logic circuit, for example as . 

implemented into a gate -array or other semi -custom logic 
circuit, for performing these functions. 

Spline-wavelet compressor 40 also includes data flow 
10 interface 166, which receives a compressed document image 
data from DCN 26 or bus 27 in process 128 (Figure 17) . 
Interface 166 provides an interface between spline- 
wavelet decompressor 40 and network 26 or bus 27, and as 
such is able to strip header and trailer information from 
15 the received data stream which, as noted above, may 

include information necessary for decompressor 40 to 
comprehend the compression techniques used in the 
compression of the received data. Interface 166 
communicates this identity and control information 
20 contained therein to other components in decompressor 40, 
by way of various control lines, some of which are shown 
in Figure 18. 

After proper receipt and reformatting of the 
25 received document data by interface 166, lossless 

decompression process 130 is next performed by lossless 
decompressor 164 . Lossless decompressor 164 may be 
implemented by way of a conventional digital signal 
processor such as the TMS320C40 or TMS320C30 available 
30 from Texas Instruments Incorporated, the i860 processor 

available from Intel Corporation, or general purpose 
microprocessors such as the 80386 and 80486 available 
from Intel Corporation or the 68030 and 68040 available 
from Motorola, programmed in such a manner as to perform 
35 lossless decompression process 130 upon the data received 
from interface 166; alternatively, lossless compressor 64 
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may be implemented as a custom logic circuit for 
providing this function. Lossless decompression process 
130 will be performed by lossless decompressor 164 
according to the technique used in the compression of the 
data (as communicated thereto by interface 166 from the 
header data) . As noted above, these lossless 
decompression techniques include Huffman encoding, 
adaptive Huffman encoding, arithmetic encoding, LSQ 
coding, and the like. The output of lossless compressor 
64 is forwarded to dequantization processor 162, for 
dequantization in process 132. 

According to this embodiment of the invention, 
dequantization processor 62 is preferably implemented as 
a programmable microprocessor or custom logic circuit for . 
performing the functions described hereinbelow; such 
implementation is believed to be readily apparent to one 
of ordinary skill in the art having reference to this 
description. As shown in Figure 18, dequantization 
processor 162 is connected to image memory banks 167 by 
way of data bus 16ld and address bus 161a, so that 
dequantization processor 162 can store the results of 
dequantization process 132 therein, and retrieve data 
therefrom in the event that additional reconstruction is 
performed. In this example, as in the case of compressor 
20, image memory 167 is arranged in four banks, totaling 
two megabytes in capacity, in process 132, 
dequantization processor 162 dequantizes the information 
in process 132 and stores the results in image memory 
167, awaiting spline-wavelet reconstruction. 

Dequantization process 132 is performed by 
dequantization processor 62 according to the quantization 
mode selected in the compression process for the 
particular document, as noted above. The particular 
quantization mode used in the compression is communicated 
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to dequantization processor 1S2 by interface 166, based 
upon information communicated in the document data 
stream. As noted above, conventional quantization 
methods that may be used in the compression process (the 
5 inverse of which is performed in process 132 of Figure 

17) include no quantization, simple thresholding, scalar 
quantization, JPEG quantization using tables, the Federal 
Bureau of Investigation quantization standard for 
fingerprint compression, and others. 

10 

As noted above, upon completion of dequantization 
process 132 by dequantization processor 162, the results 
of the dequantization are stored in image memory 157 to 
await reconstruction by digital signal processor (DSP) 

15 160. DSP 160 is the main processing unit for performing 

spline -wavelet reconstruction process 148 of Figure 17. 
Examples of modern digital signal processors suitable for 
use as DSP 160 according to this embodiment of the 
invention are the TMS320C25 and TMS320C30 digital signal 

20 processors manufactured and sold by Texas Instruments 

Incorporated; of course, other digital signal processors 
and microprocessors may alternatively be used to perform 
the spline-wavelet decomposition operations described 
hereinbelow. DSP 160 is coupled to data bus 16 Id and 

25 address bus 161a (collectively referred to as memory bus 
161) for communication with program memory 165 and image 
memory 167. According to this embodiment of the 
invention, and similarly as in the case of data 
compression described hereinabove, DSP 160 is programmed 

30 by way of code stored within DSP 160 or in program memory 
165, to perform decomposition of the compressed document 
data in image memory 167 according to pre-calculated 
reconstruction coefficients stored in program memory 165. 
During and after reconstruction process 134, the document 

35 image data is stored in image memory 167. DSP 160 is 

also connected to control bus 163, so that it may be 
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efLt t ne * ^ »« - - that it can 

process" ; ' dUri " 9 of decomposition 

egression (1 .e.. ^ ^ ~ 

oompress uT^l^" lt r thaC ^ C ° 

identified by data interface 166. The three 
reconstruction processes 134a , 134b . be 
described in detail hereinbelow 



process of process 48a described hereinabove with hioh 
pass and low-pass coef fici^ ««ove, with high- 

rprnBe , fc . coe££lc ient sequences convoluted with 

reconstruction sequences / D ) /„ l 
n «„ ^ tPk ' # i^i' respectively 

Because of the use m« ^ -, *-^v eJ .y. 

duai waveiet function , , th ? fUnCti ° n Md 

l~l c "°n the reconstruction sequences 

A . {«} e*act!y equal the decomposition sequence^Ta 7 
(M, respectively. „ . result> £he reconst ^= eS M ' 
Process X34a may aiso be done purely with 
operations, thus facilitate the decompression process 
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by modem data processing hardware of reasonable 
performance levels. 

As indicated from the description of the 
decomposition process hereinabove, the compressed 
document image data provided by deguantization process 
132 contains coefficient sequences corresponding to the 
■LL". »LH«, -HL- and "HH» decomposition components, from 
a particular decomposition level j. if, for example, the 
decomposition of process 48a for a particular document 
was done with a single pass (row convolution and column 
convolution), the coefficient sequences »LL", -lh- »HL« 
and »HH» stored in image memory 167 would correspond to 
array 94 of Figure 9 discussed hereinabove. 

Referring now to Figure 19, reconstruction process 
134a will now be described in detail. For the first 
pass through process 134a, these coefficient sequences 
will be in dual space, as noted above. Accordingly, 
process 134a begins with process 140 which, prior to 
spline -wavelet reconstruction, first transforms the . 
coefficient sequences »LL", -m-. "HL" and "HH" from dual 
space into spline space. Process 140 is preferably 
performed by way of the well-known CholesJcy computation 
method, given the duality principle as discussed above 
relative to equations (20) et seq. Subsequent passes 
through process 134a (i.e., when the result of decision 
135, described hereinbelow, indicates that the document 
was not fully reconstructed in a prior pass through 
process 134a) will not perform the transformation of 
process 140, as the coefficient sequences will already 
have been transformed into spline space. 



Upon completion of (or skipping) process 140, 
process 134a begins performing the inverse of 
decomposition process 48a described hereinabove. The 
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reconstruction of process i 3 * a ■ 

according to the recon^ ^ SffeCt °P e «tes 

convolved with reconstruct ° efflCient fences are 
<*>■ fences ( Pk) , 

will apply.. ■ L10Wln 3 "construction formula 



cr 1 = 



As noted above m„ 

resets by operatic, ^ o th ! nherenWj ' *— the 

convention procMs J * ~i- li*t., si„il arlv , 
results in an upsanpl 0 ? t Z T"™ " 4a 
equation (so) . 9 ' he results >V operation of 

^ cess 134 a are also taken ' 144 ln 

^cess x« eonvolves , \tt\e <! C ° 1U ' m "' i " "««*on. 
'«*■ co^onent. with "^-"^ ^""ion. the 

«^n=e {Pk) . whi P ch Zc:r 3tTUCti0 " °""**«t 

in process 48a h^TT^ 
<«) for the linear case th< " •'"""on 

— . ' tlUS Se9uence U) is defined 



(32) 
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The 1/4 terra may be applied to the resultant data later, 
allowing the convolution of process 142 to be performed' 
solely with integer values, in addition, the coefficient 
sequences -LL- and «LH» are inherently upsampled by a 
factor of two in the convolution of process 142, as noted 
above. The results of the two convolution operations of 
process 142 are summed into coefficient sequence {c^) . 

Similarly, process 144 is performed by way of which 
the »HL», "HH" component coefficient sequences are 
convolved, in column- wise fashion, with the 
reconstruction coefficient sequence {gj ; as noted above, 
reconstruction coefficient sequence {qj exactly equals 
decomposition sequence {bj used in the decomposition of 
the input document. For the linear case, the sequence 
{bj is specified in equation (33) as follows: 

^'^'^'^'V— (1.-6,10,-6,1), b k -0 for all other k 

(33) 

Again, the 1/24 division may be performed at a later 
time, to allow the convolution of process 144 to be 
performed using only integer values. This convolution 
will also effectively upsample the coefficient sequences 
as noted above, and the sum of the results of the two 
convolution operations of process 144 may be considered 
as coefficient sequence {d 3 * 1 }. 

The coefficient sequences {c^ 1 }, {d^ 1 } produced by 
the column-wise convolution of processes 142, 144 are 
then stored in image memory 167, in process 145. 
Referring back to Figure 9 for the example of a single 
pass decomposition and reconstruction, the coefficient 
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sequences {c j *M M^n 

92. '•«)«: thi. stage correspond to array 

Processes 146, 14 a <-u 
Which the colun^se l^ol f by way of 

convoked in the rZ^TZT^T ^ ™ 
corves sequence (e w, in ^ • " 

Composition/reconstruction dlreCti ° n " lth 
whiie process Us -^^.C^^T 
wise direction wii-h h«„ 1 ' n the row " 

M inherentuLpli 1 ;, Z^Z 'Z^" 

processes, due to the sel.-.. .> ae 
result of the convoil su ™*"°» levels. The 
summed, is the c««L ! —» 

~ - -;e~- - r - then 

sequence ( c >) at this st ao - proce « ■ The resultant 
example of a sing le pass L ** the 

Decision 135 (Pioure 171 • ! 19Ur * S - 

if the va!ue fl Perf0r, ° ed " 

case the document ii^TV* ^ '° in 
consistent witTL^nf ^ ^ "^"""ed 

described he^Latv T ™T" °' <*" ^"ssion header 
the manner ^ * 



-h *: rrjr^r^r- 4 - — 

expressed according to I a^LST^ ^ 

decomposition process 48b describe h= wavelet 

described hereinabove tat l3 ^ ^ ■ Also as 

««ve, Interpol atory wavelet 
decomposition process 4flh w avej -ec 

- hase „aveLt -^r^r^^^ 
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the decomposition sequences {aj , {b„j of equations (41) 
are used, incorporating the interpolatory spline function 
and its associated wavelet. 

As indicated hereinabove, and in contrast to the 
dual base wavelet reconstruction process 134b, the 
reconstruction coefficient sequences {p k ), {qj for the 
interpolatory approach are different from the 
decomposition sequences {a k }, {bj . Based on equation 
(42) described hereinabove, the reconstruction sequence 
(pj for the interpolatory wavelets will have p. x - h, p 0 = 
1, and Pl = with all other values of p equal to zero- 
and based on equation (37). the reconstruction sequence ' 
{qj wall have only a single non-zero element, namely q x - 
1, with all other values of q equal to zero. 

Figure 20 illustrates the operation of 
reconstruction process 134b according to the 
interpolatory wavelet technique. Process 152 retrieves 
coefficient sequences «LL", "LH", «HL» and «HH» from 
image memory 167; unlike process 134a, however, no 
transformation from dual space into spline space is 
required in this case, as no transformation into dual 
space was performed in the decomposition. 

The reconstruction of process 134b also operates 
according to the reconstruction tree of Figure 5b, in 
which the document component coefficient sequences are 
convolved with the reconstruction coefficient sequences 
{Pkh {<&} def ined hereinabove, according to the 
reconstruction formula of equation (50) hereinabove, in 
addition, just as the convolutions in decomposition 
process 48 inherently downsampled the results by 
operation of the summation limits; similarly, the 
convolution processes in reconstruction process 134b 
effectively upsample by operation of equation (50) . 
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^ t w 0 . dimensi0 L v r se t en i c n es the r-" direction f ° r 

with reconstruction r, 0 fr . components 
the linear case p "~ <*> «*• *- 
.11 other values of p equal ^ ^ «"h 
•PPlied to ehe resultant data later J? ~ 5r 
convolution of process 154 , Z * llowu ^ the 
integer values The L f ** "lely with 

convolution vJS^TZZ.^ " * ^ 
coefficient sequence (c>-M slnil ", int ° 
performed by way of v*< J\ Slmil «ly. process 15S is 
"Y way of which the "HL" »m» 

wh^h, l, iteaTcr^/r ici,nt -~ <*> 

values of , egual to J' ^ * ' L. «"h all other 
"4 may thus be nerfo™ * convolution of process 

This convolutLn^u™!":: 9 ^ 
Efficient sequence" Is^TZ^lT 
results of the two convolution e '. and the sum of the 
may be considered as 2 ^tions of process 155 

as coefficient sequence (d*' 1 }. 

The coefficient sequences / C J-M /ds»n 
the column-wise convolution { } produced 

then stored in imaT Presses 154 . 155 are 

Processes 159 ^ITT ^ ' * P ^ SS 157 ' 
- column-wise convoTu^ 

convolved in the row-wise direction » ' ' 

convolves sequence <c>-M 1„ th « . 159 

/ in the row-wise* v****- • 

reconstruction =oeffici.„t sequence Tp 1 whT 
161 convolves sequence {*•«, * t h. , """^ 
with reconstruction coefficient " direc "o 

prior convolution ^ fT"" ln ^ 

occurs in these proce^ 1 t t""' 
levels. The result of t^ 8ele " ed «— "on 

result of the convolution processes 159, 
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161 when summed, is the coefficient sequence {=,), „ hich 
xs then stored in image memory 16, i„ process 163 

if c the io " I 35 • ,Pi9Ui r 17) ia then to 

case th 3 P ° int " e9Ual " 2ero " " 

case the document image has been fuliy reconstructed and 

processing win continue in the manner described 
hereinbelow. 



c . 



Waveler, packet- ^^^^,- Hnn 



For documents that were ri~~~ mr ^„„j 
wav of th 0 * t i . decomposed and compressed JOy 

way of the full-tree decomposition using wavelet packets 

Is ZlToZeTl' - tb. compressed loZll 

is performed by way of reconstruction process i 34 c which 
also operates according to wavelet packets. Wavelet 
packet reconstruction is governed by equation (49, : 



(49) 



TJ 1 tw « "~-truetlo« process, for two-dimensional 
data, thus wxll follow in the manner illustrated in 
Figure 21, which is the reverse of the full-tree 
decomposition of Figure Ha. As before the 
reconstruction of components into a high-frequency 
component reverses the spline and wavelet sequences., so 
that an reconstructing a high pass component in the next 
level, the high-pass component will be convolved with 
spline function coefficient reconstruction sequence { Pk} 
and the low-pass component will be convolved with wavelet 
function coefficient reconstruction sequence {*} 
Similarly, reconstruction process 134c is the reverse of 
decomposition process 134c described hereinabove 
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According to this embodiment of the inv.n- • 
for the wavelet oari^ ^ invention, and 

lnn „. C P acket decomposition sequences • 

Appendix A hereto m»Hn- * «quences listed m 

P lne fun ction coefficient 
reconstruction sequence "icient 
coeffi^,. ec ^ enc e {p,} and wavelet function 

coefficient reconstruction sequence ( a \ = 
Appendix B hereto. ^quence {qj are llsted in 

Referring now to Figures 22 and « 
components of the document . 

-th Pr0 ce SE 17d . in th 7: re et: n ;r ction be3ins 

,the particuiar document is read ^ ? ^ M » f ° r 
way. depressor 40 will oT / ""rpreted; in this 
-ponents of the dL^ LT^^T ^ 
can skip thoee components thaf! """"""^ and 

compassed document. TrlT, \ » ™ * 
:: iCh MP »" "'Sieves componen ^.1^7' 

^TuT for oonvolution Klth 

described, relative to the £1o! h " "°" be 

reconstruction tree ot ^ "f^ ^ » - 
be described in the case where aH 

no pruning of th!V components are present 

course, if the tree £ ^TIT' 1 - - « ' * 

-at certain component: ~ ^1^""* 
decomposition process 4R^ 

contempxated J t*"^£ e ^"~" ~ - 

Processes 171, 173 17c n-, • • 

» * -l/5, 177 initiate th*» 

reconstruction b, convoy the components in coiumn- 
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wise fashion. Specifically, process 171 convolves 
components {x} (the blur image,, { B), { A} , ^ {c) with 
spline reconstruction coefficient sequence {Pk , in column- 
wise fashion and process 173 convolves components ( H > ( 

' {L} ' {N) With wave ^ reconstruction coefficient 
sequence (q,) in column-wise fashion. Process 174 sums 
in paired fashion, the results of processes 171, i 73 to 
complete the upper-half (lower frequency) portion of the 
array in Figure 13d; convolved components {x} and {h} are 
summed into component {x} of Figure 13d, convolved 
components {b} and {j} are S ummed into component { B) of 
Figure 13d, convolved components { A) and {L ) are summed 
into component {a} of Figure 13d, and convolved 
components {c} and {«} are summed into component { C} of 
Figure 13d. 1 ' 



As described above in decomposition process 48c 
higher-frequency components were decomposed by 
convolution of the spline and wavelet sequences in 
reverse order. Reconstruction process 134c thus must 
reconstruct higher -frequency components by convolution of 
their respective low- and high-frequency components with 
the reconstruction sequences also in reverse order 
Process 175 thus convolves components ( D} , {E} , {P) ". and 
. ™ th WaV€l6t ^instruction coefficient sequence (*} 
m column-wise fashion, and process 177 convolves 
components {l}, { K } , < M} , (o} with spline reconstruction 
coefficient sequence { Pk} in column-wise fashion. Process 
174 sums, in paired fashion, the results of processes 
175, 172 to complete the lower-half (higher frequency, 
portion of the array in Figure 13d. As such, convolved 
components {D} and {1} are summed into component {d} of 
Figure 13d, convolved components {E} and (k) are summed 
into component { E) of Figure 13d, convolved components 
IF) and {M} are summed into component {f} of Figure 13d 
and convolved components {G} and {0} are summed into 
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component {G} of Figure n 

Process „, convolves ZlT Passes 17 6 , 

freouency components ,x "d mZZTL "* * l0 " er - 
components <E} and ( G ' " Jf ' ^ hi 3"er-frequen=y 

coefficient silence ",1, " " eo —™«« 
convoives each^f lower f'^ """^ «*« 

reconstruction coefficient «o , * ' h wavele t 

result of p.j::::^:^ *'- in — 

into co.pone n r ( c7r;t~ tS ~ 
convokes component s^ent tT^ 181 

w - j^r f ~~ -~ 

process 185 convoives ^' talf -™S'> • "milarty. . 
fashion with JeLt " W ln 

<*J and Process rcroTvrc"" """"^ ~ 
row-wise fashion with spUn7 Se9USnCS <=> in 

sequence {p J . Prooeos * \™ " c ° n «™ction coefficient 

Process 184 sums the result-- 
185, 187 into component sequence l A ) of p • 
the right-half -array, . * * Fl9Ure 13b (i ' e -< 
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The reconstruction operation of process 134c is 
completed, for this level of reconstruction, by the row- 
wise convolution of component sequence {X} with spline 
reconstruction coefficient sequence { Pk) in process 186> 
and by th e row-wise convolution of component sequence (a) 
with wavelet reconstruction coefficient sequence {*} in 
process 188. Process 190 sums the results of processes 
186, 188 into the document image {X}, and stores these 
results in image memory 167. 



di Completion of rAn ons tnirf 



ion 



Referring back to Figure 17, upon completion of the 
selected one of reconstruction processes 134a, i 34b 
134c, decision 135 is performed under the directions 
data controller 156 to determine if the document has been 
fully reconstructed according to the header compression 
information described hereinabove. For example, if the 
results of the completed process 134a, 134b, 134c returns 
a document that has a compression ratio, relative to the 
input document, of u, spline -wavelet reconstruction 
process 134 is complete and decision 135 will pass 
control to process 136. m contrast, if in compression 
the decomposition process 48 was repeated to further 
compress the document, and thus if the compression ratio 
of the document image at the end of process 134 (eg at 
the first pass therethrough) relative to the input 
document is greater than i:i, spline -wavelet ' 
reconstruction is not complete, and decision 135 will 
return control to dequantization process 132 for 
repetition of the appropriate spline-wavelet process 134. 

Upon full reconstruction being reached (decision 135 
returning a YES result) , DSP 160 passes the results to 
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data controller iss « 

» - — ^rr^t'T converter 

converter 32 i s «!„,,• la , figure l) . F 0rmat 

operations o£ byte eim,™ • Perform the lnv «se 
AS discussed above ^nT f0rma " in9 £ °* di ^*X- 
byte-paOced prio^o ^li ^ ^ been 
which case byte™ £ "—eiet degression, in 

* expansion process 13« will i~ 
to recover the docunent. As such ft necessary 

Perform the necessa^p^^ M 

infection transnatted^Tth^he d T' T ^ '° 
type of byte-packina „ T ttat lnd i«tes the 

Per pi*e/o £ Lt tLf "sT ^ — »» * 
document. Pormat converter 13 T""' 

conversion process l 38 • . Performs format 

accent is'refol ; T or TsT' ■ 

appropriate for the partfliar ! t " """"^ 

graphics display prfn^ P 

dec OBp ressed foL.^c 1 °2 Ut ' ""^ *» 

then collate, and the 'data LVbT SCrUCti ° n Pr °« SS *• 

desired manner. 7 Panted in the 



Coni-1i |ff i- n 



The methods and svat„. , 
-compressinc docament^ Its^T- 

specification rh.\ «"■*»»« *>W 

benefits ^J^lT"^ *»-— « 

looaiization of the Lut 
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in addition, the document image compression and 
decompression techniques according to the preferred 
embodiments of the invention, in using spline-wavelet 
functions that have compact support, avoid boundary 
effect artifacts in the compressed documents. By 
avoiding boundary effects, the present invention greatly 
improves the accuracy of the reconstructed compressed 
document (relative to the input document) , and also 
enables parallel processing (or computing) to be applied 
in the decomposition process by cutting the document into 
small pieces and, after reconstruction, piecing together 
the full image. Using the B-spline as the scaling 
function also provides display functions such as 
magnification and compression (»zoom-in» and "zoom-out") 
m the display of the reconstructed document. The 
usefulness of the reconstructed document, and thus the 
entire system, in allowing interactive display of the 
received or retrieved document is therefore much improved: 
over conventional document compression and decompression 
systems. 



In addition, it will have been appreciated that the 
present invention allows the compression and 
reconstruction operations to be done strictly with 
integer operations. As such, both compression and 
reconstruction operations according to the present 
invention may be done more efficiently, for a given 
compression ratio, than can conventional document 
compression and reconstruction processes. The ability to 
use integer operations also enables these processes to be 
performed with moderate performance data processing 
equipment, thus allowing wide-spread use of the present 
invention, as high-performance workstations are not 
necessarily required to practice the present invention. 
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^Tl^lzr^T ation aUows f ~ — 

cy ln lts "Pleraentation. 

While ehe invention ^ describe . . 

relative to its preferred .™k „ deBcrlbed herein 
'contested thaf m^l^T"' " * 

these embodiments, ^^1, ^"^ives to. 

<*tainin 3 the advan^^T" 8 "* 
-"1 be apparent to those of oT ° f lnVentiOT " 
having ref. rence to tt ° 8kiU th ° a " 

It is contemplated t ^ „T ' " "* dr "' in9S - 
alternatives are witMn t T 1C " i0M ^ 

auhsec.entiv eUimed °* ^ « 
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APPENDIX A 



m = 2 



m = 4 



*k + t 



b k + 1 



0.683012701892 0.866025403784 



0.893162856314 



-1.475394519892 



0.316987298108 -0316987298108 



0.400680825467 



0.468422596633 



-0.116025403784 -0.232050807569 



-0.282211870811 



0.742097698477 



-0.084936490539 0.084936490539 



-0.232924626134 



-0345770890775 



0.031088913246 0.062177826491 



0.1290983571218 



-0389745580800 



0.0222758664048 -0.022758664047 



0.126457446356 



0.196794277304 



-0.008330249198 -0.016660498395 



-0.066420837387 



0.207690838380 



8 | -0.006098165652 0.006098165652 



-0.067903608499 



-0.106775803373 



0.002232083545 0.004464167091 



0.035226101674 



-0.111058440711 



10 | 0.001633998562 -0.001633998561 



0.036373586989 



0.057330952254 



11 | -0.000598084983 -0.001196169967 



-0.018815686621 



0.059433388390 



12 1 -0.000437828595 0.000437828595 



■0.019473269356 



■0.030709700871 



13 I 0.000160256388 0.000320512777 



0.010066747520 



•0.031811811318 



14 [ 0.000117315818 -0.000117315818 



0.010424052187 



0.016440944687 



15 | -0.000042940569 -0.000085881139 



-0.005387929819 



0.017028029466 



16 1 -0.000031434679 0.000031434678 



-0.005579839208 



-0.008800839839 



17 | 0.000011505891 0.000023011782 



0.002883979478 



-0.009114745138 



18 0.000008422897 -0.000008422897 



0.002986784625 



0.004710957034 



19 1-0.000003082990 -0.000006165980 



0.001543728719 



0.004878941541 



20 I -0.000002256905 0.0000022569054 



-0.001598768083 



0.002521687975 



21 0.000000826079 0.0000016521587 



0.000826326663 



0.002611601542 
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WHAT IS ry^iMRn T| c . 



1. A method of compressing digital data 
representative of a document, comprising the steps of- 
formatting digital data corresponding to an 
image of the document into a two-dimensional array of 
digital values, each value corresponding to the 

r™:: the document at a — - - cation of 

dioifi f erf0rming 3 first ^composing of the array of 

I hll ' ln a first direction ' into 

and hxgh-freguency portions using a decomposition scaling 
function and a decomposition wavelet function 
respectively, said decomposition scaling and wavelet 
functions not requiring support outside of the array 
Q * , • OITnin9 3 SeC ° nd decom P° s ing of the results 

usina th I' d6COmPOSing °^ in • ™ direction, 
using the decomposition scaling and wavelet functions; 

communicating the results of the decomposing 
steps to a receiving unit. 

2. The method of claim 1, wherein said 
communicating step comprises: 

a memory. St ° rin9 ^ ° f ^ steps in 

3. The method of claim 1, wherein said 
communicating step comprises: s 

transmitting the results of the decomposing 
steps to a decompression system. 
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4. The method of claim 3 , further compris 

transmiteed resulce i t r s :j;r dlr r :i t tr ccin9 of 

reconstruction scaling function and . e " 10n ' usln * » 
wavelet function ..J , reconstruction 

^unction, said sealing and wavel.r *„.,....• 
requiring support outside of the aJay ^ 

reconstn,cLTs"7in a t h th :. reSUltS ° f ^ 
reconstruction 9 scaL, ^'.ST""' ^ °" 
wavelet function; and reconstruction 

«- re=onst~ft: p r" Ual ~« - * — - 

recoJ; ? 6 meCh ° d ° £ CUinl 4 ' Whe " i » <*• 
reconstruction scaling function is equal to the 

decomposition scaling function, and ^herei! th! 

reconstruction wavelet function is equaT 0 the 

decomposition wavelet function. 

6- The method of claim i * ^ 

oi ciaxm l, further comprising- 

values into P 7: u r Jpat t^T^ ° f 
decomposition scaling t haVi " 9 a *»* 

and wherein said first- 
are performed using ^Z^ZllZTZllZ ^ 
action and the dual decomposition wave!et f^L. 
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and wherein the decomposition wavelet function is 
the wavelet function corresponding to the centralized B- 
spline scaling function. 

8. The method of claim 1, wherein the decomposition 
scaling function and decomposition wavelet functions are 
based upon non-orthogonal, symmetric, spline-based 
wavelet packets. 



9. The method of claim l, wherein the first 
decomposing step comprises: 

(a) decomposing the array of digital values, ir 
the first direction, into low- frequency and high- 
frequency portions using the decomposition scaling 
function and the decomposition wavelet function, 
respectively; 

(b) decomposing the low- frequency portion from 
step (a) , in the first direction, into low-frequency and 
high-frequency portions using the decomposition scaling 
function and the decomposition wavelet function, 
respectively; and 

(c) decomposing the high-frequency portion from 
step (a) , in the first direction, into low- frequency and 
high-frequency portions using the decomposition wavelet 
function and the decomposition scaling function, 
respectively. 



10. The method of claim 9, wherein the second 
decomposing step comprises : 

(d) decomposing the results of the first 
decomposing step, in the second direction, into low- 
frequency and high-frequency portions using the 
decomposition scaling function and the decomposition 
wavelet function, respectively; 

(e) decomposing the low-frequency portion from 
step (d) , in the second direction, into low-frequency and 
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10 



15 



lt!l e lTL P °r i0nS USin9 d «^^ion scaling 
respect" t/" 0 ^" 10 " "^^ 

«- «ep ir t T s B f 9 , the hi9h - fr ^-=V portion fro. 



frequency co^ ^"ST?"' 

the lo„- freguency decomposition i„ ZTf ° £ 

the f irst deceasing sTep dire " i0n of 

hi^ency Cne~; ^I^onl * 
of the low-freau»i,rv ^ second direction, 

« «- «r S /:e™::x s ; tlOT in the £irst di — 

the high-fresuency decomposition^ J dl "««". •* 
<* the „« decomposing^; 0 ^" ^««io„ 

h^-frecuency ZZT^ '° 

directs of the £irst decomposlng 

»nd further comprising- ' 

to coTr:::^^- hl - and - =— — 

12 ' a T ?t e er th °! ° f Clalm 5 ' fUrCh6r 

co^unicatin" step B e^r tiZin? ^ "* Pri0r <° 
the ll. LH HI, and !T 9 l0Ssle " Session on 

^ HL, and HH components. 
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13. The method of claim 1, wherein said formatting 
step comprises: 

substituting at least first and second digital 
values for display attributes corresponding to the image 
of the document, to generate a substituted array of the 
at least first and second digital values; and 

after said substituting step, byte-packing the 
substituted array. 

14. The method of claim 13, wherein said 
substituting step first, second and third digital values 
for display attributes corresponding to the image of the 
document . 

15. A method of displaying compressed document 
image data, comprising: 

receiving compressed image data corresponding 
to a document and storing the compressed image data in 
memory, said compressed image data arranged as blur and 
high-frequency coefficient sequences; 

in a first image direction, applying a 
reconstruction scaling function and a reconstruction 
wavelet function to the compressed image data to 
reconstruct a low-frequency component and a high- 
frequency component; 

in a second image direction, applying the 
reconstruction scaling function and the reconstruction 
wavelet function to the reconstructed low- frequency 
component and high-frequency component to reconstruct the 
document image ; 

displaying the reconstructed document image. 

16. The method of claim 15, further comprising: 
repeating said steps of applying the 
reconstruction scaling function and the reconstruction 
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wavelet function in the first an H « 
direction Second im *sre 



directions 

17 



The method of claim is 

Derfnrm . , m 15 ' furt her comprising.- 

Performing lossless decompression on ^ 
compressed document image data , pri * r ^ " ^ . 
steps. *«-«-»r to said appl ying 

18. The method of claim k ..u 

19. The method of claim k fc u 

cxaim is, further comprisina- 

function and the d!al I *«itl« scaling 

=> the dual decomposition wavelet function. 

20- The method of claim is, wherein t-h. 
reconstruction scaling function sTltraiLd B , • 
scaling function; centralized B-spline 

«- watlet^tit lr nSt ™ C raVel " *• 

3 P iine acai^rfirtC ^ ondin9 to the centraii - a - 

21- The method of claim is, wherein th* ~ 
Pluralities of coefficient sequences, 

the ete P « fixing the reconstruction ' 



convolving the reconstruction scaling * «. . 
with the first plurality of coefficient ^ 
first direction; coefficient sequences, in the 
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10 convolving the reconstruction wavelet function 

with the second plurality of coefficient sequences, in 
the first direction; 

after the convolving steps, summing each of the 
first plurality of coefficient sequences to an associated 
15 one of the second plurality of coefficient sequences, to 
produce third and fourth pluralities of coefficient 
sequences ,- 

after the summing step, convolving the 
reconstruction scaling function with the third plurality 
of coefficient sequences, in the first direction; 

convolving the reconstruction wavelet function 
with the. fourth plurality of coefficient sequences, in 
the first direction; and 

then summing each of the third plurality of 
25 coefficient sequences to an associated one of the fourth 
plurality of coefficient sequences, to produce fifth and 
sixth pluralities of coefficient sequences; 

and wherein the step of applying the reconstruction 
scaling and wavelet functions in a second direction 
30 comprise: 

convolving the reconstruction scaling function 
with the fifth plurality of coefficient sequences, in the 
second direction; 

convolving the reconstruction wavelet function 
35 with the sixth plurality of coefficient sequences, in the 
second direction; 

after the convolving steps, summing each of the 
fifth plurality of coefficient sequences to an associated 
one of the sixth plurality of coefficient sequences, to 
40 produce a low- frequency component and a high-frequency 
component of the document image; 

after the summing step, convolving the 
reconstruction scaling function with the low-frequency 
component of the document image, in the second direction; 
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with the jr^ 9 ^ reCOn8t -«-n wavelet function 

lcn cne ^9h- frequency component of t-h*» ^« 
in the second direction, and ~ U " ent 

f«Hn. n then SUmmin9 ^"V"** «d high- 

t ::rin c :r:: ts to produce an — — - - 

informs " inPUt SOUrCe f ° r -«9ital image 

information corresponding to a document to be 

communicated; and 

compressor circuitry having an input coupled to 
said input source, for decomposing each frame of digital 
image information in a first image direction, by 

iateger ^ image 

«vZt f USln9 3 8Calin3 £UnCti0 ° Md « corresponding 
the convolution interval, into a first low-frequency 

thin:::,^ 3 first hi * h -*~« ~^rjr,or 

Mqh frl deCO,nPOSin9 "= h <* low-fr.ou.ncy and 

high-frequency components of each frame of digital video 
«». information in a second image direction b" 
performing integer convolutions of the first low- 

flrtt IL , ""Ending wavelet function, into 

first and second pairs of low-frequency and second high- 
frequency components, said first pair being the 
decomposed representation of said first low-frequency 
component and said second pair corresponding to* 
decomposed representation of said first high-frequency 
component, said compressor circuitry also havingT 
output for presenting the ,i rst and second pairs of low- 
frequency and high-frequency components. 



23 



The system of claim 22. further comprising, 
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a format converter circuit coupled between said 
input source and said compressor, for converting the 
digital input video information into a portable gray- 
level format. 



24. The system of claim 23, wherein the format 
converter circuit comprises circuitry for byte-packing 

source . 



ri x, co "rcuicry tor byte -packing 
data corresponding to image data received from the input 



25. The system of claim 24, wherein said input 
source provides color document information; 

and wherein said byte-packing circuitry performs 
byte-packing of data containing more than one bit per 
picture element of the document. 

26. The system of claim 22, wherein said compressor 
circuitry comprises: 

decomposing circuitry for performing the 
decomposing of each frame of digital image information 
into the first and second pairs of low- frequency and 
high-frequency components; 

quantization circuitry for quantizing the high- 
frequency component of said first pair, and for 
quantizing the both the low-frequency and the high- 
frequency component of the second pair; and 

lossless compression circuitry, having an input 
coupled to said quantization circuitry, for performing 
lossless compression of the first and second low- 
frequency and high-frequency pairs, prior to presenting 
the decomposed frames of digital video image information 
at the output of the compressor circuitry. 

27. The system of claim 26, further comprising: 

main controller circuitry, for controlling the 
operation of the decomposing circuitry, so that the low- 
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frequency component of said first pair may be 
repet iti ly provided ^ ^ decomp 

further decomposing. ^ 

cir=u!trv f SySte '" ° f Claim " hereln 8Sid lessor 

circuitry comprises: 

an image buffer coupled to the input of the 
pressor circuitry for storing a frame of digital video 
image information; and <"3«al video 

a digital processor for performing the 

in'toTf" 3 ° £ eaCh £ra " e ° £ dlSltal ^ formation 
into the f lrst Md secMd pairs Qf lQw . fr 

high-frequency components by performing integer 
convolutions of the digital image information using a 
Zl Z IT ^ 3 CO ""«* -velet function 
i™ reqU " e SUPPO " ° UCSide ° f «- convolution 



interval . 
29. 



The system of claim 28, wherein said digital 

zzri r p r ates according to ^ ^uj^ 

precalcuiated matrices corresponding to spline and 
wavelet function coefficient sequences stored in said 
compressor circuitry. 

30. The system of claim 19, further comprising- 

a communications network coupled to the output 
of the compressor circuitry; 

to «,M deC ° mPreSSOr = ir cuitry. having an input coupled 
to said communications network, for reconstructing the 
decomposed digital image information in the second image 
direction by performing integer convolutions of the 

TIT 1 in£ormation * reconstruction scaling 

function and a corresponding reconstruction wavelet 
function that do not retire support outside of the 
convolution interval, for then further reconstructing the 
digital image information in the first image direction by 
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performing integer convolutions of the digital image 
information using the reconstruction scalL .J , 
functions; and waling and wavelet 

means for displaying the reconstructed frames 
of digital image information. 

31. A system for displaying compressed image data 
corresponding to a document, comprising: 

• memory tor storing the compressed image data- 
decompressor circuitry for reconstructing the ' 
compressed image data in a first image direction by 

LforTtT lntSger C ° nVOlUtionS ° f «" ^tal image 
information using a reconstruction scaling function an* 
corresponding reconstruction wavelet t^£^£^ 
require support outside of the convolution inte^al and 
for t h en further reconstructing the compressed i^ ^ 
m a second image direction oy performing integer 
convolutions of the digital image information using the : 
reconstruction scaling and wavelet functions; and 

means, coupled to said decompressor circuitry 
for outputting the reconstructed compressed image dat Z' 

coupled'to 7116 SyStem ° f CUim 31 ' WherSin Said «— ry is 
coupled to a communications network. 

33. The system of claim 31, wherein said 
decompressor circuitry comprises: 

a digital processor coupled to said memory for 
reconstructing the dicrital i ma «« memory, for 

s «« aigicai image information usino 
precalculated matrices of integers 
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BOUNDARY -SPLINE- WAVELET COMPRESSION FOR VIDEO IMAGES 



* * * * 



This invention is in the field of video image storage 
and communication, and is more specifically directed to 
compression and decompression of video images, including 
motion video images and high-quality still video images. 



Background o-F the Inve»ni--irm 

2^ Conventional vid eo eomnrf»Hai rm 

Modem electronic data processing systems are now 
often used not only for conventional numeric and text 
processing tasks, but also for processing data 
corresponding to visual images. As such, many modern 
computer and telecommunications systems have "multimedia" 
capabilities, where the data processed and communicated 
includes video images either generated by a computer or 
digitized from a conventional video camera. 

The capacity required to communicate video images on 
a real-time basis is huge, however, when measured against 
modern capabilities. For example, a single video image 
(i.e., frame) displayed by a rectangular array of picture 
elements ("pixels") arranged in 640 rows by 800 columns, 
with the color of each pixel represented by twenty- four 
digital bits, would require over 1.52 million bytes (1500 
kbytes) of digital memory to store all information in the 
frame. While this memory requirement for a single video 
frame is large, digital storage of a series of frames, such 
as a motion picture, will quickly consume the disk storage 
capacity of even high-end computers and workstations. 
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The large amount of digital data necessary to 
represent a video frame not only impacts computer storage 
requirements, but limits the rate at which conventional 
systems can communicate motion pictures. Considering that 
conventional high-speed digital communication channels have 
a bandwidth of on the order of from 4 0 to 80 MHz, it 
becomes quite apparent that conventional motion pictures of 
thirty frames per second, with 1500 kbytes of digital 
information per frame, cannot be fully transmitted in real- 
time over state of the art digital communications systems. 

In response to these limitations on the ability of 
modern computer systems to store and communicate video 
information, various types of data compression techniques 
have been developed in recent years. Conventional data 
compression techniques are generally referred to as of 
either "lossless" or "lossy", depending upon whether data 
is discarded in the compression process. 

A survey of conventional lossless data compression is 
provided in Simon, "Lossless Compression: How it Works", PC 
Magazine (June 29, 1993), pp. 305-13. Examples of 
conventional lossless data compression techniques include 
Huffman encoding, Pano-Shannon encoding, and arithmetic 
encoding, as well as dynamic variations of the Huffman and 
Fano-Shannon probabilistic encoding techniques. In 
general, lossless compression techniques are primarily used 
to compress entered data such as data generated from data 
processing operations, rather than sampled data 
representative of analog video or audio signals, as 
decompression will reproduce all bits of the original 
compressed data stream. 

Lossy data compression techniques, in contrast, 
provide additional data compression efficiency over 
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lossless data compression, as some amount of data is 
discarded in these techniques. As a result, lossy data 
compression techniques are generally used on sampled data, 
as some amount of inaccuracy relative to the true input 
data is tolerable; lossy data compression is, of course 
inappropriate for use on entered data streams such as those 
produced by a data processing operation. Accordingly, lossy 
data compression techniques are widely used in the field of 
compression of video and motion picture images to obtain a 
high degree of compression, as some inaccuracy may be 
tolerated. A survey of conventional lossy data compression 
techniques may be found at Simon, -How Lossy Data 
Compression Shrinks Image Piles", PC Maaa^o (July 1993) 
pp. 371 et seq. 

A popular conventional lossy data compression 
technique is referred to as the JPEG (Joint Photographic 
Experts Group) method. A description of this technique may 
be found in Barnsley and Hurd, Fractal t-^ c^.c^. 
(AK Peters, Ltd., 1993), pp. 219-228. The JPEG compression 
method initially divides the image into blocks of pixels 
and a Discrete Cosine Transform (DCT) is performed on each 
Pixel block, producing a representation of the block as 
coefficients corresponding to frequencies and amplitudes 
rather than corresponding directly to color information 
These coefficients are then quantized, or rounded off and 
a deference algorithm is performed over all quantized 
blocks in the image, in a selected scan order. This 
difference algorithm subtracts a DC term corresponding to 
the mean pixel value of a block, from the DC term of the 
preceding block. The difference coefficients are then 
scanned in a different order, such as a zig-zag order, and 
the non-zero coefficients (i.e., blocks in which a 
difference from the preceding block occurred) are coded to 
indicate the number of preceding zero coefficients (i e 
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the number of pixel blocks in which no change occurred) and 
also the value of the non-zero difference. Lossless 
compression is then often applied to the coded result to 
further compress the data. Decompression is performed by 
reversing the compression process, producing the 
displayable image. 

While the JPEG conventional video image compression 
technique is useful in obtaining high degrees of 
compression, it has been found that JPEG compression is 
incapable of being used in a real-time fashion for a motion 
picture. This is because the time generally required to 
perform the JPEG decompression of a motion picture frame 
exceeds the display time for the frame (1/30 second), and 
as a result the motion picture image cannot be decompressed 
for real-time display. Temporally accurate display of a- 
motion picture compressed according to these techniques, 
thus requires the decompression and display to be done in 
two steps, with the decompressed motion picture stored on 
video tape or another medium from which the motion picture 
can be played with the proper time base. 

Another conventional method of lossy video image 
compression referred to as Recursive Vector Quantization 
(RVQ) quantizes the pixel blocks directly, without a DCT or 
other transform, according to a set of selected reference 
tiles. £gg Simon, July 1993, op. cit. . The reference tiles 
are selected according to an iterative technique, based 
upon the accuracy of the results relative to the original 
image. As noted in the Simon article, compression 
according to the RVQ method is computationally intense and 
complex, but decompression can be done quite rapidly. 

Another type of conventional lossy video image 
compression techniques is . referred to as fractal 
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compression. As is well known in the art, a fractal is a 
mathematical image object that is self -similar, in that the 
image can be represented in terms of other pieces of the 
image. In fractal image compression, the input image is 
similarly divided into pixel groups, or tiles. Each tile 
is then approximated by a transformation (contractive, 
rotational, or both) of one or more other reference regions 
of the image. The compressed image thus consists of a full 
representation of the reference region, plus the 
transformation operators for each of the tiles. Each tile 
of the image is decompressed by performing a transformation 
of the reference region using the stored transformation 
operator for that tile. Detailed descriptions of 
conventional fractal image compression techniques and 
systems for performing the same may be found in Bamsley & 
Hurd, Fractal Image Co mpression (AK Peters, Ltd., 1993), in 
U.S. Patent No. 4,941,193, and in U.S. Patent No. 

5, 06b, 447. 



2^ Frequency and time windowin g functions 

By way of further background, the field of wavelet 
analysis has recently become popular in the analysis of the 
time and frequency response and behavior of signals. The 
following section of this application is intended to 

provide a theoretical background for wavelet analysis 
techniques in order to both convey the state of the art in 

wavelet analysis, and also to provide the necessary 

background for the person of ordinary skill in the art to 

fully appreciate the present invention. 

In the general sense, wavelet analysis is concerned 
with performing time -frequency localization of the signal 
to be analyzed (i.e,. the "input signal"). Time -frequency 
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localization refers to the analysis of a portion of the 
frequency spectrum of the input signal over a selected time 
window. As will become apparent from the description in 
this specification, time -frequency localization of an input 
5 signal enables data processing techniques to be applied to 
the signals for a wide range of purposes. 

a. Conventional analocr filtering 

10 

In the time domain, frequency-windowing is done by 
convolving a time-domain window filter function with the 
input signal; in the frequency domain, the frequency- 
windowing is done by multiplying the spectrum of the input 

15 signal with the frequency- domain transfer function of the 
filter function. Typical filter functions include low-pass 
filters (e.g., the Shannon sampling function) and band-pass 
filters. Through use of such filters, a bandwidth limited 
signal f^(t) (i.e., zero amplitude for all frequencies 

20 above a limit £2) may be decomposed into the sura of a low- 
frequency component with a series of non-zero frequency 
bands. An expression for such a decomposition is as 
follows : 

f Q(t) =f Q, Wo (t) + g Q ,i(t) +. . .+g Q/N (t) 

25 

tl] 

where is the low-pass filtered (Cd<6)0) component of 

the input signal fjj(t), where g Q ^(t) is the band-pass 
3 0 filtered signal for the ith frequency band, and where (0^ = 
fl. An ideal low-pass filtered component f Q corresponds 
to the time -domain convolution of the well-known Shannon 
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sampling function with the input signal. Ideal band-pass 
filtering may be performed by time-domain convolution of a 
filter function of the type: 

sinco^t-si na)^ t 

Tit 



[2] 



with the time-domain input signal, co n andfi/p^ being the 
upper and lower limits of the frequency band. 

Each of the low-pass and band-pass filter functions 
provide ideal frequency localization of the input signal 
fjj(t) , such that each expression and g Q ±{t ) provide 

precise information regarding the frequency' spectrum of 
input signal f fl (t) within its respective frequency band. 
However, the time localization provided by this 
decomposition is quite poor, as these filters do not 
provide very precise information about when (in the time 
domain) a signal behaves differently within certain 
frequency ranges. As many important real-world signals 
include brief periods of time of rapid transient change, 
analysis of signals decomposed in this manner will not be 
able to account for the time at which such transient 
behavior occurs. For signal analysis where time 
localization is important, conventional Fourier analysis 
techniques therefore falls short of the need. 



b. Time and frequency windowing 



Certain conventional signal analysis techniques have 
addressed this problem by time -windowing the input signal, 
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thus allowing time- localization as well as frequency 
localization. According to these techniques, a window 
function h(t) is applied to the input signal f (t) to window 
the input signal near a specified point in time t=b. This 
windowing may be considered by the integral transform: 



ffit)h(t-b) dt 



L3j 

where the bar over the function h(t-b) denotes complex 
conjugation. For a real- valued even windowing function 
h(t), this windowing process corresponds to convolution, 
allowing treatment of the windowing function h(t) as a 
filter. For example, if h(t) is the Shannon sampling 
function, the windowing process of the above equation will 
apply a low-pass filter to the input signal. However, it 
has been observed that the slow decay of the Shannon 
sampling function over time results in a very imprecise 
time-domain windowing operation. 

Those windowing functions h(t), for which the square 
of the magnitude have finite first and second moments and 
finite energy, (i.e., that decay sufficiently rapidly at 
infinity) will produce a time-window having a "center" t 
and a -radius" A h . The center t* may be calculated as I 
mean value, while the radius may be calculated as s the 
standard deviation of the windowing function around the 
mean t*. For a windowing function of radius A h , the width 
of the time -window will be 2A h , commonly referred to as the 
RMS duration of the windowing function h(t). in the 
frequency domain, if the square of the magnitude of the 
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Fourier transfer* h(«) of the windowing function h(t) has 
fimt. first and second moments, the frequency domain 
windowing function fi(«) wiU ^ a center ^ ^ g width 

2Afr calculated in a manner similar to the mean and 
standard deviation (doubled) of the frequency domain 
function the width 2A fi is usually called the RMS 

bandwidth of the windowing function h(t) . If the RMS 
duration A h is f±nlte , then the time-domain windowing 
function h(t) is a time-window; similarly, if the rms 
bandwidth 2A h is finite, the frequency-domain windowing 
function f]{b>) is a frequency window. 

Referring back to the ideal low-pass and band-pass 
filter time-domain functions noted above, it will be 
readily apparent that their first moment is infinite 
meaning that neither of these ideal filter functions can 
serve as time windows if used as windowing functions h(t) . 
However, the frequency domain representations of the ideal 
low-pass and band-pass filter functions have finite RMS 
bandwidth aAfi. so that these filters provide ideal 

frequency- localization as is evident from their ideal 
nature . 



As discussed above, accurate analysis of real-world 
signals containing transient behavior requires both time- 
localization and frequency-localization. The Uncertainty 
Principle has identified those windowing functions h that 
provide both time windowing and frequency windowing as 
those functions that satisfy the following inequality 



A h A h * * 

[5] 
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It has also been previously found that the only types of 
windowing functions h(t) that achieve the lower bound of 
the Uncertainty Principle are those of the. form: 

h{t) =ce J ' at e (t - 2>)2 /4o 

[6] 



for some constants a, b, c and a with a > 0 and c * 



0, 



Further indication of the presence of a time -frequency 
window by any windowing function h(t) may be obtained 
through the Parseval identity. The generalized windowing 
function h(t) noted above corresponds to: 

ao 

CD 

I fit) hit-h) dt~-±- 1 f (©) e^STofdo 



[7] 



With reference to the left side of equation [6] , the time 
window is given by: 



[b + t" - A„, b + t* + Aj 



[8] 



such that the time window that is centered on t = t* is 
shifted by the parameter b; the time window also has a 
radius of A h in the time dimension. Similarly, with 
reference to the right-hand side of equation [s] , the 
frequency window is given by: 



[9] 
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and is thus centered at A/=6> 4 „ it h width 2Afi. Figure la 
illustrates the location of time -frequency window 2 of 
equation [6] in a time -frequency coordinate system. 

For causal real-valued window functions, where: 

=£(«)" 

the function \f\(cu)\ is an even function, so that the center 

is located at a=o and the frequency window of equation 
[S] becomes 

l " A h' A tf [io] 

However, while the time -frequency window of the filtering 
function h(t) may be moved along the time axis of Figure la 
by . changing the value of b, the window is fixed in 
frequency at the center frequency or, in the case of 
real-valued even functions as noted above relative to 
equation [10], is fixed at a center frequency CO, = 0 as 
shown in Figure lb. This fixation in frequency of ' the 
time-frequency window in limits the usefulness of the 
windowing process of equation [3] . 



— Th e Short-Tims *Vy^-iq r Tr^.fn^ (STE-T) 

Rudimentary Fourier theory indicates that translation 
in the time-domain corresponds to a phase-shift in the 
frequency domain, and conversely that a phase-shift in the 
time-domain corresponds to a translation in the frequency 
domain. Accordingly, a phase shift in the windowing 
process of equation [3] should allow for sliding of the 
frequency window along the frequency- axis . 
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Considering a real -valued window function 0(t) which 
serves as a low-pass filter (i.e., 0(fl=O) = 1) , and for 
which 0(t), |t| M 0(t) and t<p(t) are in L 2 , the short-time 
Fourier transform of (pit) is defined by: 

00 

(G^f) (b, S ) : =| f ( t) e-^ c <|> ( t-b) dt 

— OM 

[11] 

for f 6 L 2 . The short-time Fourier transform (STFT) of 
equation [11] is also referred to in the literature as the 
Gabor transform. Applying the Parseval identity results in 
the following expression for the STFT, from which the 
center and radii of the time and frequency windows are 
apparent : 

00 

(G^f) (b. I) f f (») -5 )d» 

7U J 

— oo 

[12] 

The short-time Fourier transform of equations [11] and 
[12] provide the improvement over the simple time -windowing 
process of equation [3] in that the frequency window 
function <p(U) is able to slide along the frequency axis. 
The frequency window for a windowing function <j>((t)) that 
otherwise (i.e., for £=0) has its center 10+ at ft/=0, is now 
localized to frequencies near &) » £ : 

a;] [13] 
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Similarly, where the center t* of the windowing function 
(pit) is also otherwise (b=0) located at the origin, the 
time window is now localized near t=b as follows: 

[b - A„ b + A,J [14] 

The STFT thus allows for sliding of the time- frequency 
localization windows both in time and in frequency, merely 
by changing the values of the phase-shift factors b, £, 
respectively. Figure lc illustrates the position in time- 
frequency space of two time -frequency windows 5 Q , 5 
having phase-shift factors (b Q , $ Q ) , < bi> 
respectively. As a result, the STFT allows the lowpass 
window function to perform bandpass filtering by changing 
the value of However, as is evident in Figure lc, the 

widths of the windows are invariant with changes in time- 
shift factor b or frequency- shift factor $ . Accordingly, 
while analysis is improved through use of the short-time 
Fourier transform, inaccuracies due to undersampling have 
been observed for those transient periods of time in which 
rapid changes (i.e., amplitudes over a wide range of 
frequencies) exist. Accordingly, not only is the ability 
to slide the localization windows in both time and 
frequency desired, but it is also desirable to allow for 
scaling of the window widths as a function of time or 
frequency. 
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3 . Wavelet analysis 



a . Theory 

Wavelet analysis techniques address the need for time- 
frequency localization windows of flexible size, primarily 
by introducing a scale parameter that is mapped to the 
frequency variable. As a result, the scale parameter will 
change the widths of the windows as a function of 
frequency/ thus changing the aspect ratio of the windows 
with changes in frequency. Since the Uncertainty Principle 
requires the area of the windows to remain above a certain 
value, the time window width decreases and the frequency 
window width increases proportionally as the center 
frequency £ increases. The narrowing of the time window 
and widening of the frequency window for high frequency 
environments more precisely detects and analyzes these high 
frequency portions of input signals. 

The basic wavelet transform is known as the integral 
wavelet transform, or IWT. The IWT includes a scale 
parameter a- in its definition as follows : 



[15] 



As such, the window function If/ narrows with changing values 
of the scale parameter a, such that the time width of Ifl 
decreases with decreasing a. The windowing function ifl (t) 
used in the IWT of equation [15] is to be real-valued as 
before, but the IWT constraints also require Ifl (t) to be a 



eifiitfiuiiL cuiir /mar 



WO 95/19683 



PCT/US95/00563 



15 

bandpass filter rather than a low pass filter, such that 
its Fourier transform *ft,. 0 ) = 0, stopping at least zero 
frequency components of the signal. Since the windowing 
function ^(t) is real-valued, its Fourier transform 
satisfies : rv ' 



[16] 



so that |p (« ) | is an even function. Because only 
nonnegative frequencies are of interest, . and since *( t ) is 
a band-pass filter, the Fourier transform j (ft, ) need only 
be considered as a frequency window in the frequency domain 
[0, ») , with the centers and widths of the frequency window 
function f («) being modified as a refiult ^ a window . ng 

function If, (t) in! 2 , for which |t|*( t ) and * (t) are also 
in L such that ft> (t) is real-valued, and where j (ft, =0 ) - o 
the one-sided (i.e., nonnegative) frequency window center 
» + as a function on the domain [o, « } is defined as: 



0) + : = 



. = o 



/!♦(©) | 2 dG> 



[17] 



and the one-sided radius of lp{id) ± B defined 



as: 
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AJ: = 



\ 



j (G)-G>:) 2 |<|f (G)) | 2 d0) 

0 _ 

QO 

/|$(o>) | 2 dO) 



[18] 



This allows the generation of an integral wavelet 
transform (IWT) using a normalization factor a"* based upon 
the scale parameter a, which scales the time-width of the 
window as a function of frequency. For a windowing 
function $ (t) that satisfies the conditions for equations 
[17] and [18] above, the IWT is defined as follows: 



yfal * 



[19] 

For the IWT of equation [19] , the bandpass window-function 
<Mt) is commonly referred to as the analyzing wavelet. 

As is known in the wavelet analysis field, and given 
the foregoing discussion, it is important that the integral 
wavelet transform W^ allows for frequency localization 
where the width of the time window is mapped to the 
frequency domain, and in which the frequency window can 
slide along the frequency axis. For finite -energy real- 
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valued input signals f(t), and since 'f (t) is real, the 
following relationship holds: 

f ( - w ) e^tjr (-ao» =jf(o>) e-^ u ^mST 

[20] 

Through the Parseval identity, one can then derive the IWT 
as follows : 



w 

(«r,f) .(2), a) -&Ref£ <») e^r, (a (o>--^ ) ) do, 

0 3 

[21] 

for all f £ L 2 R , where Ui + * is the one-sided center of 0 (a; ) 
on the domain [0, » } , ^ where ^ ±B defined ag fdllQwag 



n (o>) :=$(o)+cot) 



[22] 



As noted above, it is desirable to map the scale 
parameter a to the frequency at which the time-frequency 
window is to be localized. Accordingly, the scale 
parameter a is mapped to the shift frequency parameter £ as 
follows : 
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a 



for some c > 0 f where c is a calibration constant. 
Substituting for the scale parameter a defines the IWT as 
follows : 



(*£f) (2>,£) : = (W t f) (b f a)=(W }jf f) (jb, -|) 

[24] 

It is convenient to set c=d) + * , so that: 

(Kf) ib. S) =^§Ref£ (o>) (a (o-$) ) do 



[25] 

This produces a frequency window i;(a«tl^)) that slides 
along the frequency axis with the value of having a 



range; 



[26] 
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The width of this window thus increases at higher 
frequencies £ , as reflected in smaller values of the scale 
parameter a. In terms of the scale parameter a, the width 
of the frequency- window is as follows: 



[27] 



where the frequency shifting term { -«//«, such that the 
frequency-width of the frequency window increases with 
increasing frequency { (decreasing values of a) . Along the 
time-axis, the time window of the IWT of equation [19] is 
given by: 



[28] 



As a result, the width of this time-window is 2a^, which 
decreases at higher frequencies £ (and lower values of a) , 
and which increases at lower frequencies For the 

transform of equation [15] , Figure id illustrates three 
time-frequency windows 6 Q , 6^ 6 2 , with varying translation 
factor pairs (b Q , * 0 >, (b^ ^) , (b 2 , ^) , respectively. 
As is evident in Figure Id, both the time-width and 
frequency-width of windows 6 vary with varying shift 
frequency such that the time-width decreases and the 
frequency-width increases with increasing £ . 
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Accordingly, it should now be apparent to those in the 
art that wavelet analysis is capable of providing increased 
accuracy analysis of signals, particularly those including 
transient components, considering that, for higher 
frequency analysis, the width of the time window decreases 
and the width of the frequency window increases. This 
ensures adequate sampling of the input signal function, and 
also allows for determination of the exact time in the 
signal at which the transient event occurred. 

As is known in the art of wavelet analysis, however, 
the definition of the proper wavelet function Ifl (t) is of 
great importance. Various specified analyzing functions 
have been used in wavelet analysis, with the selection of 
the function made according to computability, or according 
to attributes of the signal under analysis. 

k» Prior wavalPt^p 

Wavelet signal analysis has been applied to signals 
produced in seismic exploration for oil and gas, as 
described in U.S. Patent No. 4,599,567. This reference 
describes a wavelet analysis technique using Morlet's 
wavelet as the analyzing wavelet. Morlet's wavelet is a 
sinusoid limited by a Gaussian probability envelope to a 
finite duration; the envelope may or may not be skewed 
toward either the leading or lagging edge of the time- 
domain envelope of the wavelet, as desired for the type of 
signal under analysis. This reference also discloses 
circuitry for performing the wavelet analysis of an 
incoming signal using such analyzing wavelet. 

Another finite duration analyzing wavelet was proposed 
by Yves Meyer. This analyzing wavelet is substantially a 
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finite duration uniform magnitude level over the wavelet 
window, analogous to the Shannon sampling function. 

Other wavelets have been proposed which are not 
expressible by a mathematical formula, but instead are 
utilized as numeric wavelets. These wavelets include the 
Battle-Lemanne wavelet, which is a spline function of 
infinite duration, and the Daubechies wavelet, which is a 
fractal wavelet of finite duration. The lack of explicit 
formulae for these wavelets limit their applicability for 
rapid and accurate computer implementation of the wavelet 
analysis technique in computer hardware. 

Another previously published wavelet is the Chui-Wang 
wavelet, which is a wavelet of finite duration but which 
may be expressed in an explicit formula. 

The support for each of the prior wavelets noted above 
is over an unbounded interval. However, since real-world 
problems require the application of the wavelets to bounded 
intervals, wavelet analysis of input signals using these 
prior wavelets result in errors of the type commonly 
referred to as "boundary effects". 

Figure 2 graphically illustrates the reason for 
boundary effects arising from such conventional wavelets 
Conventional wavelet 7 illustrated in Figure 2a is first - 
order spline-wavelet that is based on a function ? having 
moments with the following properties (for i - o l 
1/ and some m a l) : 



WW 
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where a, b are scaling parameters as discussed above. The 
conventional wavelet 7 of Figure 2a is not orthogonal over 
a bounded interval [c, d] , however, meaning that: 



/ 



[30] 

for all integers i > o. 

Figure 2 graphically illustrates the performance of 
the IWT using wavelet 7 having the above noted properties 
at a point in the time series which happens to be at or 
near the boundary of an interval [c, d] . Data points f 



f 



3 / f 4 correspond to input signal samples within the 
interval, which are plotted against wavelet 7 in Figure 2; 
in this example, for purposes of explanation, the input 
signal sample data closely matches the shape of wavelet 7 
within the interval [c. d] . The position of wavelet 7 
corresponds to the position, in performing the IWT, of the 
sample point of interest at the boundary value t=a. Since 
wavelet 7 at this position requires support outside of the 
interval [c, d] for which input signal data exists, the 
zero values that must be assumed for the data points f , f 
outside of interval [c, d] necessarily fail to match 
wavelet 7. This will result in an non-zero result for the 
IWT, even where the input data signal within the interval 
[c, d] exactly matches wavelet 7. As is well known in the 
field of signal processing, this inaccuracy due to wavelet 
7 requiring support outside of the bounded interval [c, d] 
is made manifest by boundary effects at the edges of the 
bounded interval, since the unbounded wavelet 7 fails to 
accurately represent the series of actual input signal 
sample data. 
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In the field of video image compression and analysis, 
boundary effects greatly affects the quality of the image 
displayed after compression and decompression. This is 
because the boundary effects will appear as false image 
data at the edges of pixel blocks corresponding to bounded 
intervals, and also at the edges of the image (even if no 
subdivision of the image is performed) . The inaccuracies 
due to boundary effects also limit the ability to magnify 
an image when displayed, considering that the magnification 
will make the boundary effect errors to become even more 
visible. 



c . Th e boundarv-spline-wavelet: 

By way of further background, a bounded interval 
wavelet is described in Chui and Quak, "Wavelets on a 
Bounded Interval", Numerical Methods of ATmmvin.aHnn 
Theory, Volume 9 (Dec. 1992), pp. 53-75, incorporated 
herein by this reference. This wavelet, which has an 
explicit formula, is not only a function of the time 
variable t, but is also a function of the position of the 
sample within the interval [c, d] , so as to account for 
boundary effects. In effect, sample locations near the 
boundaries of the interval will correspond to different 
wavelet shapes than will sample locations within the 
interval that are away from the boundaries. Boundary 
effects are eliminated as the boundary wavelets do not 
require support outside of the interval. 

Referring now to Figures 3a through 3d, an exemplary 
set of four first-order wavelets 8 according to the Chui- 
Quak boundary- spline -wavelet approach are illustrated. 
Figure 3a illustrates the shape of "boundary" wavelet 8 
for a sample location near the boundary t=a of the interval 
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[a, b] , while Figure 3d illustrates the shape of boundary 
wavelet 8 b for a sample location near the boundary t=b of 
the interval [a, b] . Figures 3b and 3c each illustrate the 
shape of -inner" wavelets 8. for sample locations within 
the interval [a, b] away from the boundaries. As is 
evident from Figures 3a through 3d, boundary wavelets 8 
8 b , have different shapes than inner wavelets 8. (which 
have the same shape as one other). As is further 1 evident 
from Figures 3a through 3d, neither inner wavelets 8. nor 
boundary wavelets 8 a , 8 b require support outside of the 
interval [a, b] , or: 



b 

/ 



Gt 
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[31] 



for i = 0, 1, . . . , m-l, and for some m a 0, for the 
entire set of wavelets 8 of Figures 3a through 3d. 
Accordingly, application of the set of boundary-spline- 
wavelets 8 to actual real-world data, for which the. time 
interval is necessarily bounded, will not produce boundary 
effect artifacts. 

Other boundary-wavelets are known, as described in 
Daubechies, "Two recent results on wavelets : Wavelet bases 
for the interval and biorthogonal wavelet diagonalizing the 
derivative operator". Recent Adv ances in w ave let A^l y ^ a 
Schumaker and Webb, ed. (Academic Press, 1993), pp. 237-58 '. 
These wavelets are not spline functions, and do not have 
explicit formulae. As a result, it is believed that these 
wavelets are of limited effectiveness in video image 
compression and decompression. 
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±j Objects of the invention 

It is therefore an object of the present invention to 
apply wavelet analysis tools to the task of video image 
compression for storage and transmission. 

It is a further object of the present invention to 
provide an apparatus for performing video image compression 
according to a selected wavelet. 

It is a further object of the present invention to 
provide an apparatus for receiving a compressed video image 
and to decompress the same for real-time playback of the 
stored or communicated video image information. 

It is a further object of the present invention to 
provide such a method of compression and decompression such 
that the compressed image may be played locally at real- 
time . 

It is a further object of the present invention to 
provide such a system and method which allows for high 
accuracy magnification of the decompressed image, with much 
reduced incidence of edge effects. 

It is a further object of the present invention to 
provide such a system and method which provides a high 
degree of compression. 

It is a further object of the present invention to 
provide such a system and method which can utilize dynamic 
compression on a frame-by- frame basis, such that high 
frequency frames may be compressed to different ratios than 
low frequency frames. 
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It is a further object of the present invention to 
provide such a system and method which facilitates 
interactive display of a motion image, including insertion, 
editing and repetitive display. 

It is a further object of the present invention to 
provide such a system and method which provides the ability 
for slow display systems to skip certain frames so that a 
motion picture can be displayed on a real-time basis by 
such slower systems, although with fewer frames per second. 



It is a further object of the present invention to 
provide such a method and system which allows for division 
of an image into several portions for purposes of 
compression and communication or storage, with subsequent 
display of the full image after decompression. 

It is a further object . of the present invention to 
provide such a method and system which allows for higher 
lossy compression ratios by further quantization, as useful 
in compressing and decompressing high-quality 24 -bit still 
images . 



Other objects and advantages will be apparent to those 
of ordinary skill in the art having reference to the 
following specification, together with its drawings. 
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Summary of th e Invent- -ii-m 

The present invention may be implemented into a method 
and apparatus for compressing and decompressing video 
images. The compression/decompression system receives 
video image data in a suitable format, such as eight -bit 
Portable Grey Level format, and includes circuitry for 
decomposing each component of the image into low and high 
frequency components by way of boundary-spline-wavelet 
decomposition. The system also includes circuitry, for 
quantizing the high frequency portions of the 
decomposition. The decomposition may be repeated until the 
desired compression ratio is obtained. Lossless 
compression may then be performed on the compressed image 
following which the compressed data may be stored for 
archival purposes, or may be transmitted to a receiving 
station. 

According to the present invention, in decompression 
mode, the system performs lossless decompression of the 
stored or communicated image data, according to the 
compression scheme utilized. Reconstruction of the image 
is then performed by way of the boundary- spline -wavelet 
approach, in a reverse fashion relative to the image 
compression. The result of the decompression is then 
formatted as desired for display. 



WO 95/19683 



PCT/US95/00563 



28 

Brief D escription of the Drawings 



Figures la through Id are time -frequency plots 
illustrating the concept of time -frequency windowing, as 
described in the background of the invention. 

Figure 2 is a graphical illustration of a wavelet with 
non-zero moments of any order relative to the interval [c, 
d], as applied to an input signal, and which is the source 
of boundary effects. 



Figures 3a through 3d are graphical illustrations of 
an exemplary set of boundary-spline-wavelets as used in the 
15 preferred embodiment of the invention. 

Figure 4a is an electrical diagram, in block form, of 
a system for producing and commiinicating or storing 
compressed video image information according to the 
20 preferred embodiment of the invention. 

Figure 4b is an electrical diagram, in block form, of 
the image format converter in the system of Figure 4a. 
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Figure 5 is an electrical diagram, in block form, of 
a compressor according to the preferred embodiment of the 
invention . 

Figure 6 is a flow diagram illustrating a method of 
compressing video image data according to the preferred 
embodiment of the invention. 

Figure 7 is a detailed flow diagram illustrating a 
method of performing the boundary- spline -wavelet 
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decompression according to the preferred embodiment of the 
invention . 

Figure 8 is a series of frequency domain plots 
illustrating the decomposition of the video image according 
to the preferred embodiment of the invention. 

Figures 9a through 9c are matrices illustrating the 
effect of decomposition of the video image according to the 
preferred embodiment of the invention. 

Figure 10 is a flow diagram of a preferred matrix 
algebra technique for performing the boundary- spline - 
wavelet decomposition of Figure 7. 

Figure ll is a chart illustrating the relationship of 
the matrices in the process of Figure 10. 

Figures 12a through I2e illustrate an example of a 
video image decomposed according to the preferred 
embodiment of the invention. 



Figures 13a through 13c illustrate the quantization of 
the decomposed video image of Figures 12a through I2e 
according to the preferred embodiment of the invention. 

Figure 14 is a diagram of the construction of a 
compressed video image frame for transmission according to 
the preferred embodiment of the invention. 

Figure 15a is an electrical diagram, in block. form, of 
a decompressor according to the preferred embodiment of the 
invention . 
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Figure 15b is an electrical diagram, in block form, of 
a format converter useful in the decompressor of Figure 



15a 



Figure 16 is a flow chart of a process for 
decompressing video image data according to the preferred 
embodiment of the invention. 

Figure 17 is a detailed flow chart of a process for 
reconstructing video image data that were decomposed 
according to the boundary-spline-wavelet approach of the 
preferred embodiment of the invention. 

Figure 18 is a chart illustrating the relationship of 
the matrices as used in a preferred matrix algebra routine 
for the reconstruction of Figure 17. 

Figure 19 is an electrical diagram, in block form, of 
a decompressor system according to an alternative 
embodiment of the invention. 
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Detailed Description of the P^f g - r red EmhnH^o^ 
— Video image data communication Ry afpni 

Referring now to Figure 4a, the construction of a 
video communication system 8 according to the preferred 
embodiment of the invention will now be described in 
detail. m this embodiment, system 8 is specifically 
designed for communicating compressed motion picture data; 
it is of course contemplated that system 8 may 
alternatively be used to communicate compressed data 
representative of a series of still video images. m 
addition, as will be described in further detail 
herexnbelow, system 8 may alternatively or also be used for 
archival storage of motion picture or still video image 
data, and the subsequent retrieval and display thereof. 

As shown in Figure 4a, in the context of video data 
communication, the transmission end of system 8 includes 
vadeo source 12 and compression system 10, while the 
receiving end of system 8 includes decompression system 20 
and video display 26. As illustrated in Figure 4a, video 
source 12 may include digital video source I2d which may be 
a conventional device such as a CD-ROM drive, scanner 
digital electronic network connection, or similar unit, or 
alternatively may be a computer storage unit such as a disk 
that contains digital video information. Video source 12 
may also or instead include analog video source 12a, which 
may be a video camera, VCR unit, television broadcast or 
cable receiver, or another conventional source of analog 
video information. m any case, video source 12 provides 
digital or analog signals indicative of the images to be 
communicated or stored by the system of Figure 4a. 
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Communication network system 15 is a conventional 
analog transmission or electronic digital communications 
network, or both an analog and digital network when 
including the appropriate clusters, analog- to-digital and 
digital-to-analog converters, and other necessary 
apparatus. Network 15 may be realized according to any 
conventional technology, including hard- wired cable, fiber 
optic cable, broadcast or satellite transmission, and the 
like. It is further contemplated that network 15 may be 
implemented merely by the physical transportation of 
portable media 22' such as floppy diskettes, CD-ROMs and 
the like. Regardless of the implementation, network 15 is 
connected between compression system 10 and the input of 
decompression system 20 to communicate compressed video 
image data therebetween. 



It is specifically contemplated that the video data 
transmission effected from compression system 10 may be of 
the broadcast type, such that a single transmission end 
(source 12 plus compression system 10) may communicate 
simultaneously or in sequence to multiple receiving ends 
(decompression system 20 and display 26). For example, 
compression system 10 may be located at a television or 
movie studio, or at a local cable television system "head- 
end", with multiple receiving ends (decompression system 20 
and display 26) located at homes, offices, or local 
theaters, depending upon the particular transmission being 
effected. 

Figure 4a also illustrates the optional use of system 
8 in the context of the archival storage of motion picture 
or still video image data, and its later retrieval and 
display thereof. Disk storage unit 22 is coupled to 
receive the compressed video data from compression system 
10 for storage. An example of disk storage unit 22 is a 
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large disk (e.g., having capacity on the order of 1000 
Gigabytes) / alternatively, or in addition to the large disk 
unit, disk storage unit 22 may be associated with a 
mainframe or supercomputer . to provide services such as on- 
line collection of video images into a library form. Disk 
storage unit is also coupled to decompression system 20, 
which is operable in this context to receive compressed 
video data therefrom for display on video display 26. The 
subsequent retrieval of the archived information in disk 8 
may be made via a digital communications network (such as 
network 15 described hereinabove) , or alternatively by way 
of a portable data storage medium such as a tape or 
diskette . 

Compression system 10 includes data format converter 
14 that is functionally connected between video source 12 
and compressor/coder 16. Format converter 14 is of 
conventional construction as used to convert the format of 
the video image data from video source 12 into the format 
suitable for compression by compressor 16. In the example 
described hereinbelow, the format utilized by compressor 16 
is eight-bit Portable Grey Level (PGM) , although fewer than 
eight bits may be used in the PGM format, depending upon 
the available hardware devices and architecture. For color 
images, the format is RGB -PGM, where each of the red, green 
and blue color components is separately stored and 
communicated in PGM format. The PGM image format will be 
described in further detail hereinbelow. Accordingly, 
format converter 14 converts the image data from video 
formats such as PCX, IMG, GIF, TIF, RLE, NTSC, PAL, and the 
like into PGM or RGB -PGM format. Of course, if the output 
from video source 12 is already in PGM or RGB -PGM format, 
format converter 14 is unnecessary, and compressor 16 may 
directly receive the data from video source 12. 
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Referring now to Figure 4b , the construction of format 
converter 14 according to the preferred embodiment of the 
invention will be described in detail, for the example 
where the output provided by format converter 14 is in RGB- 
PGM format. As noted above, format converter 14 can 
receive video data from either analog or- digital video 
sources 12a, I2d, respectively, and as such two input ports 
are provided to format converter 14 along with two 
functional devices to reformat the received data into the 
desired format. 

In the example of Figure 4b, digital video image data 
presented by digital video source 12d is received by format 
decoder and color expansion device 21. The format decoder 
portion of device 21 decodes the data from the format 
presented by source 12d, such format being PCX, IMG, TIF, 
GIF, RLE, YUV, etc., into a color matrix, or color palette 
table, representation of the image signal. Format decoder 
circuitry is well known in the art, such as described in 
Rimmer ' Supercharged Bitmanr^H r.~, r v,^ (Windcrest/McGraw 
Hill). This decoding is followed by a color expansion 
operation performed by device 21, such that the output 
therefrom is in raw RGB format, with each pixel represented 
by successive bytes of red, green and blue color 
components. Conventional circuitry for performing such 
color expansion is readily available in the field, 
including such devices as color palette RAMs . The output 
of format decoder and color expansion device 21 is 
presented to color separator 25. 

On the analog side, analog video information is 
presented by analog video source 12a to analog screen 
grabber and digitizer device 23 . Device 23 first converts 
the analog input data from its NTSC or PAL format, by the 
screen grabber capturing the input screen data. Device 23 



WO 95/19683 



PCT/OS95/00563 



35 



then digitizes each pixel of the captured screen, in the 
conventional manner. Conventional circuitry for performing 
the screen grabbing and digitizing functions may be used to 
implement device 23. The output of device 23 is also in 
the raw digital, or binary, RGB format as presented by 
device 21. 

Color separator 25 is a conventional digital filter 
for separating the raw binary RGB signal presented by 
either of devices 21, 23 into RGB - PGM format. As is well 
known in the art, the RGB -PGM video data format includes, 
for each pixel, three eight-bit components. These three 
eight -bit components correspond to the intensity of the 
red, blue and green colors, respectively, to be displayed 
for that pixel. Of course, other byte widths may be used 
to digitally represent the intensity of each color 
component . 

The RGB -PGM format used in this preferred embodiment 
of the invention decomposes the input video image into 
three screen representations, one for each of the red, blue 
and green color components. Each pixel location in the 
input image is thus represented by a pixel value in each of 
the red, blue and green decompositions, indicating the 
intensity of that color component at that pixel location of 
the image. In the eight -bit PGM format, the intensity of 
each color component for each pixel can thus range from 0 
(black, or no intensity) to 255 (full intensity) . The 
eight-bit PGM format' is particularly convenient v for 
processing by microprocessors or computers using ASCII 
coding and programming environments, as the fundamental 
storage unit is the byte in these environments. 

The three channels of RGB- PGM data produced by color 
separator 25 is then presented to compressor 16, either 
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sequentially or in parallel, for compression as will be 
described hereinbelow. According to this embodiment of the 
invention, each of the R, G, B components are compressed, 
decompressed, and otherwise processed separately from the 
other components for that image. 

Of course, if the data presented by video source 12 
specifically digital video source l2d, is a i ready in RGB '_ 
PGM format, format converter 14 is not necessary in the 
system of Figure 4a. 

Compressor 16 includes the necessary compression 
circuitry, an example of which is described in detail 
hereinbelow, for compressing the formatted digital data 
according to the boundary-spline-wavelet technique that is 
also described in detail hereinbelow. compressor 16 may 
also mclude coding circuitry for formatting the compressed 
data into the suitable format for communication over 
network 15 (or storage in disk 22) . *s will also be 
described in further detail hereinbelow, specific 
information regarding attributes of the communicated or 
stored information may be specifically included in the data 
coded by compressor 16. 

Similarly, decompression system 20 includes 
decompressor 18 coupled to network 15 (or to disk 22 as 
the case may be) . Decompressor 18 receives the transmitted 
or stored image data, reformats it into a form suitable for 
decompression, if necessary, and decompresses the data 
Decompressor 18 in this embodiment of the invention 
communicates the decompressed image data to format 
converter 24, which converts the decompressed data to the 
appropriate format for display by display 26. 
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As illustrated in Figure 4a, display 26 may be 
implemented as a digital display 26d, to which digital data 
may be directly applied thereto; alternatively, display 26 
may be implemented as a conventional analog display, with 
the appropriate NTSC or other analog video format data 
applied thereto. According to this example, the output 
data from decompressor 18 is in PGM or RGB -PGM format, and 
thus format converter .24 will convert the PGM data into 
PCX, IMG, GIF, TIF, RLE, NTSC, PAL, RGB, or another display 
format. Of course, if digital display 26d is used and is 
capable of receiving and directly displaying PGM format 
data, format converter 24 is unnecessary. 

■2-! Constru ction of the compressor 

Referring now to Figure 5, the construction of 
compressor 16 according to the preferred embodiment of the 
invention will now be described in detail. It is, of 
course, contemplated that other architectural arrangeit^nts 
of circuitry may be used in the compression of video data 
according to the present invention. Specifically, it is 
contemplated that a conventional general purpose computer 
system may be capable or. performing the compression of the 
video image data according to the present invention. 
However, the example of Figure 5 incorporates a preferred 
embodiment of the circuitry for performing the data 
compression functions to be described herein. 

In this example, compressor 16 preferably includes one 
main RGB -PGM channel controller 28 and three substantially 
identical channel compressor subsystems 29 R , 29 , 29 
Three channel compressor subsystems 29 R , 29 29 are 
provided according to this embodiment of the invention, 
considering the separation of the input video data into the 
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three component channels by format converter 14 described 
hereinabove. Compressor 16 also preferably includes data 
flow interface 39, for receiving data trom each ot channel 
compressor subsystems 29 and presenting the same to network 
5 15. 

Main controller 28 receives the three channels of rgb- 
PGM data from format converter 14, and forwards each 
channel of data separately to the appropriate channel 
compressor subsystem 29 Rf 29 Q , 29g. Alternatively, 
compressor 16 may be implemented to have only a single 
channel compressor subsystems 29 which processes each 
channel of data sequentially, in which case main controller 
28 would control the sequential forwarding of image data to 
the single compressor subsystem 29. 

Main controller 28 is connected by way of the 
appropriate bidirectional buses and ' control lines to 
control the functions within compressor 16. In addition to 
its control of subsystems 29, main controller 28 also 
controls the timing, feedback and sending operation of 
compressor 16, including control of the data flow interface 
39. As such, main controller 28 is preferably a general 
purpose programmable microprocessor or other central 
processing unit (CPU) of sufficient computational power and 
capacity to process some or all of the image data and to 
control the performing of the image compression functions 
to be described hereinbelow. It is contemplated that 
microprocessors having performance levels similar to, or 
30 greater than those of the 80486 type (including those 
available from Intel Corporation or Cyrix Corporation) , of 
the 68 040 type (including those available from Motorola) , 
and of the SPARC processor type (available from Texas 
Instruments Incorporated or Sun Microsystems, inc.) will be 
35 suitable for use as main controller 28 in compressor 16. 
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The construction of channel compressor subsystems 29 

will now be described in detail relative to the 

construction of subsystem 29 R as shown in Figure 5; it is 

contemplated that the other subsystems 29„, 29^, will be 

* ■ KG* 
similarly constructed. Each channel compressor subsystem 

29 according to this embodiment of the invention is 

specifically designed to perform the functions of boundary 

spline wavelet decomposition, quantization and lossless 

compression, as used in the compression operation according 

to this embodiment of the invention. 

Channel compressor subsystem 29 according to this 
embodiment of the invention includes the circuit functions 
of digital matrix process 30, timing circuit 37, 
quantization processor 32, lossless compressor 34, program 
data memory 35, and multiple memory banks 36 for storage of 
image data. 

Timing circuit 37 performs the functions of receiving 
the PGM format channel data from main controller 28 and 
forwarding the received data to memory banks 36 in its 
subsystem 29. In addition, timing circuit 37 controls the 
timing and feedback among the other functional components 
of channel compressor subsystem 29, including the matrix 
operations performed by digital matrix processor 30, 
quantization performed by quantization processor 32, 
lossless compression performed by lossless compressor 34, 
and accesses to memory 35, 36. 

Digital matrix processor 30 is a processing circuit of 
conventional architecture that is specifically suitable for 
performing vector and matrix operations as used in the 
decomposition of image data according to the preferred 
embodiment of the invention, as will be described in detail 
hereinbelow. As will be apparent from the description 
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below, these operations include the retrieval of pre- 
calculated matrices from program data memory 35, the matrix 
operations utilized in performing boundary-spline-wavelet 
decomposition of the video channel data, and storage of the 
results in memory banks 35. Examples of presently 
available components suitable for use as digital matrix 
processor 30 include digital signal processors such as the 
i860 processor available from Intel Corporation and the 
TMSC40 digital signal processor available from Texas 
Instruments Incorporated, and also general purpose 
microprocessors such as those of the 80386 and 80486 
families available from Intel Corporation, and of the 68030 
and 68040 families available from Motorola. 

Quantization processor 32 is a logic circuit for 
filtering the data corresponding to decomposed images in 
order to achieve the desired compression ratio. it is 
contemplated that conventional processing circuitry or 
custom logic circuitry for performing this function in the 
manner described hereinbelow, will be readily apparent to 
one of ordinary skill in the art. According to this 
preferred embodiment of the invention, quantization 
processor 32 may be selectably controlled to perform the 
quantization process according to various selectable modes 
These modes are selectable by way of quantization mode 
register 31, which stores a three digit code defining the 
type of quantization to be performed by quantization 
processor 32. An example of the codes storable in 
quantization mode register 31 and their corresponding 
quantization modes are as follows: 



WO 95/19683 



PCT/US95/00563 



10 



15 



20 



25 



30 



35 



0 
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No quantization 
thresholding 
scalar quantization 

JPEG quantization (i.e., using tables) 
reserved for other quantization modes 
(e.g., vector quantization) 



Lossless compressor 34 may be implemented by way of a 
conventional digital signal processor such as the TMSC40, 
programmed in such a manner as to perform lossless 
decompression upon the results of the quantized output from 
quantization processor 32. The lossless decompression 
performed by lossless decompressor 34 is according to the 
desired conventional technique, such as Huffman encoding, 
adaptive Huffman encoding, arithmetic encoding, LSQ coding, 
and the like. Alternatively, lossless compressor 34 may be 
implemented as a custom logic circuit for providing this 
function. The output of lossless compressor 34 is 
preferably compressed data for the channel (R, g, B) in 
bitstream format, for application to data flow interface 



Data flow interface 39 provides an interface between 
compressor 16 and network 15, and as such must gather the 
bitstream output from lossless compressors 34 in each of 
the subsystems 29 and arrange the same into a suitable 
format for transmission. Data flow interface 39 also 
provides a feedback signal to main controller 28 upon 
transmission of a frame of compressed data, based upon 
Which main controller 28 may commence the processing of the 
next image frame to be compressed. 

The preferred format in which, the output compressed 
data from data flow interface 39 is communicated over 
network IS will be described in further detail hereinbelow. 
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This example of compressor 16 is intended to support 
the compression of high definition real-time true color 
video image data, where "true color" indicates the use of 
twenty- four bits of color information for each pixel, 
resulting in 16.7 million possible colors. The frame rate 
for compressor 16 is intended to be on the order of thirty 
frames per second, so as to support "real-time" video image 
compression. 

As noted above, if the color and frame rate 
requirements are reduced from real-time true color video, 
it may be possible to implement compressor 16 as a single 
channel, i.e. with a single channel compression subsystem 
29. In this implementation, color data could be compressed 
sequentially for the R, G and B components of the RGB -PGM 
input data, under the control of main controller 28. 



3. 



. Boundary-flpline-wavftlet virion imaaf> riai- a conrore.Qsi nn 



Referring now to Figure 6, a method of compressing 
video image data according to the preferred embodiment of 
the invention will now be described in detail. it is 
contemplated that the architecture of compressor 16 of 
Figure 5 and described hereinabove is particularly suitable 
for the performing of this method, although it is further 
contemplated that other computer architectures and 
arrangements may alternatively be used to perform the 
process of Figure 6. 

The flow chart of Figure 6 corresponds to the 
compression of a single frame of video image data. 
Accordingly, for the compression of a motion picture, the 
process of Figure 6 is performed sequentially for each 
frame of the motion picture. In the case of still image 
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compression, of course, the process of Figure 6 is 
performed for each image. 

The process of Figure 6 begins with the conversion of 
the video image data for the frame into the desired format 
for compression which, according to this preferred 
embodiment, is the well-known PGM format. As discussed 
above relative to Figure 4a, this conversion is preferably 
performed by format converter 14,. allowing compressor 16 to 
be dedicated to the compression process. As noted above, 
the PGM (Portable Grey Level) format expresses each picture 
element ("pixel") as a numerical value corresponding to its 
brightness; for eight -bit PGM, the values range from 0 to 
255. Color images may be expressed in a mode referred to 
in the art as RGB-PGM, where a PGM image is provided for 
each of the red, green and blue color components of the 
image. It is believed that the RGB-PGM format is the most 
adaptable format for processing of color image data by way 
of the present invention, as it directly provides "true- 
color display information for high performance display 
systems, and is also readily adaptable for conversion to 
color look-up table ("color palette") display systems. 

After conversion by format converter 14, the RGB-PGM 
image data is separated into separate R, G, B channels by 
main controller 28. This allows each of channel 
compression subsystems 29 to compress the image data in the 
manner described hereinbelow in parallel with one another. 
Of course, if a monochrome image is being compressed, only 
a single subsystem 29 is necessary; alternatively, if the 
compression rate allows, a color image may be compressed by 
sequentially compressing the separate R, G, B channel image 
data. 
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^ — Boundary- spline - wavelet decompositHrtn 

After channel separation by main controller 28, a 
frame of PGM image data is stored in memory banks 36. in 
subsystem 29 in a row-based order, arranged from top left 
to bottom right of the image. Each PGM image frame is then 
processed by boundary- spline-wavelet decomposition, 
indicated by process 40 of Figure 6. Figure 7 is a 
detailed flow chart of process 40 according to the 
preferred embodiment of the invention, to which attention 
is now directed. The compression method of Figure 7 will 
be described relative to a single channel (R, G, or B) , as 
it will be understood by those of ordinary skill in the art 
that a full color image will be compressed by the parallel 
or sequential compression of the other color channels in 
similar fashion. 

The boundary- spline-wavelet decomposition of process 
40 may operates upon individual blocks of pixels of the PGM 
frame, such as blocks of pixels that may be as small as 
eight -by- eight pixels, or as large as on the order of 1024- 
by-1024 pixels or greater. Process 48 indicates that the 
operation of each subsystem 29 will be performed upon the 
PGM frame in subdivided blocks. One of the important 
benefits of the present invention is that the size of the 
blocks defined in process 48 depends primarily upon the 
architecture of subsystems 29 in compressor 16 and of 
digital matrix processor 30 therein, but is not dictated by 
considerations of picture quality. This is because the 
boundary- spline wavelets used in the decomposition 
eliminate inaccuracies in the displayed data resulting from 
boundary effects. 

Alternatively, if the computing capacity of digital 
matrix processor 30 and subsystems 29 is adequate, an 
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entire frame may be decomposed in process 40 without 
dividing the image into smaller blocks of pixels. The 
present invention will still provide important benefits due 
to the elimination of . boundary effects and inaccuracies 
around the edge of the image that would otherwise be 
present if unbounded wavelets were used. In addition, 
regardless of whether the image is divided into smaller 
blocks, the elimination of boundary effects according to 
the present invention also enables magnification of the 
image to be performed after decompression, without the 
limitations and artifacts that would otherwise be present. 

Following division of the frame into the appropriate 
image blocks, the decomposition of process 40 continues 
with processes 50, 52 in which boundary-spline-wavelet 
decomposition is performed upon the image block, according 
to this embodiment of the invention. As will be described 
in detail herein, according to the preferred .embodiment of 
the invention, the decomposition of process 40 is performed 
utilizing the wavelets described in the Chui and Quak paper 
cited hereinabove and incorporated by reference hereinto, 
and illustrated in Figures 3a through 3d discussed above. 
Process 50 first decomposes each horizontal row of the 
image block into two equal numbers of components of low- 
frequency (low-pass) and high-frequency (band-pass) , and is 
followed by process 52 in which each vertical column of 
each of the low- frequency and high-frequency results of 
process 50 are again decomposed into two equal numbers of 
components of low- frequency and high-frequency. 

Referring now to Figure 8 in combination with Figures 
9a through 9c, decomposition processes 50, 52 will now be 
described in further detail. Figure 8 illustrates a 
frequency domain representation of function f^, and which 
corresponds to image block 51 of Figure 9a that represents 
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the discrete function f N (x,y) . The discrete function 
f N^ x, y' is a functional representation of image block 51 in 
PGM format, such that each value "x" in Figure 9a is a 
digital value corresponding to image brightness at a 
position (x,y). In this example, image block 51 is eight 
pixels square. 

As shown in Figure 8, process 50 of Figure 7 
decomposes^the function into a low-frequency or low-pass 
component f N -i and a high-frequency or band-pass component 
According to the preferred embodiment of the 
invention, process 50 performs such decomposition for each 
horizontal row of image block 51, so that the result of 
process 50 is a matrix 53 that includes, for each row, a 
low frequency portion f H . 1 (x,y) and a high frequency 
portion g N . 1 (x,y). m other words, the decomposition of 
process 50 is performed considering the image data for each 
row of image block 51 as a one-dimensional spatial function 
in the x-dimension. 

As discussed hereinabove, the decomposition of process 
50 is performed according to boundary- spline -wavelet 
decomposition techniques, so that boundary effects are 
eliminated in the resulting frame. Accordingly, for each 
horizontal ^ row of image block 51, the low frequency 
component (or f^^y) in the spatial domain) in the 
corresponding row of decomposed image block 53 is a spline 
interpolation of the spatial data of image block 51. The 
high frequency component g^-l (or g N _ 1 (x,y) in the spatial 
domain) in the corresponding row of decomposed image block 
53 corresponds to the boundary wavelets applied to the 
spatial data of that row of image block 51. 

Referring back to Figure 5, according to the preferred 
embodiment of the invention, it is contemplated that the 
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decomposition of process 50 is performed by digital matrix 
processor 30 by way of a matrix algebra technique. 
According to this matrix technique, the decomposition of 
process 50 is intended to calculate the matrices c N_1 , d N_1 
according to the following relationship: 



[32] 

where c N_1 and d N_1 are matrices of coefficients of the 
low- frequency and high-frequency components of the input 
signal, respectively. 

Attention is directed to Figures 10 . and 11 in 
combination for a detailed description of this technique. 
Figure 10 is a flow chart illustrating the matrix algebra 
procedure for performing processes 50 and 52 of Figure 7 
(the algebra being the same in each case) . Figure 11 is a 
chart illustrating the relationship among the decomposition 
matrices used in performing processes 50, 52 according to 
the flow chart of Figure 10. 

As shown in Figure 10, the first step of the 
decomposition begins with the process 60 which interpolates 
the function f, for each horizontal row, using a B-spline 
function 0(x), to provide a matrix representation {c} N of 
the input image block 51 using the following relationship: 



[33] 
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where m is the degree of the spline, where k is the number 
of sample points within the interval (i.e., the length of 
the row of image^ block 51, in pixels), and where the B- 
spline function <p^ (x) is of the form: 

=2 *$(2J*-k) 

[34] 

According to the preferred embodiment of the invention, it 
has been observed that the accuracy of the decomposition 
and the stability of the result of odd degree splines 
(i.e., linear, cubic, etc.) is improved relative to even 
degree splines (i.e., quadratic). As such, it is preferred 
that m, in equation [31] be an odd number (m = 1, 3, 5, . 
. . ) . 

Upon determining the matrix c N , process 62 (Figure 10) 
is then next performed to compute the coefficients of a 
matrix {a} N , using the relationship a N = C^c 1 *. According 
to this preferred embodiment of the invention, the matrix. 
C 17 is the inner product of the cardinal spline function 

[35] 

The calculation of process 62 returns the matrix a N , as 
shown in Figure 11. 
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Process 64 is then performed in order to compute 
intermediate decomposition coefficients utilizing the 
spline and boundary- wavelet concepts. These intermediate 
decomposition coefficients are also obtained by way of 
matrix algebra using digital matrix processor 30 of 
compressor 16, shown in Figure 5. For the low- frequency, 
low-pass , or spline component , the intermediate 
decomposition coefficients a N_1 are calculated using the 
following matrix operation: 

= (p^-1) T a N 



[36] 



where P 1 *" 1 is a known matrix that contains coefficients of 
the B- spline to be applied to the input image block, and 
which has the following form: 



[37] 



The elements p of the matrix of equation [37] are defined 
as follows: 



k+m-1 dl 



Bl*t k+m-i) 

[38] 

for 1 from -m+1 to -l, and for k from m+21 to -m+l, where 
B ° m ,i represents the B-spline of order m, on the zeroth 
level and with ith initial knot. 
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For the high-frequency, band-pass, or wavelet 
component of the row of the image block 51 , the 
intermediate decomposition coefficients b N " X are calculated 
using the following matrix operation: 



[39] 

i. N-l . 

where Q is a known matrix containing coefficients of the 
boundary-wavelets to be applied to the input image block, 
and having the form: 

[40] 

where the elements q of the matrix are defined as follows: 



J « i 2 



B£ k (k+m-i) 



[41] 



for 1 from -m+1 to -1, and for k from 3m-2+2l to -m+1, and 
where B° m i represents the B-spline of order m on the 
zeroth level and with ith initial knot. In equation [41], 
the values i 2 , i 3 , i 4 are defined as follows: 

1 2 = min (3m-2+21, k+m-2) 

1 3 = max (-m+1, k-m) 

1 4 = rain (2(l+m-l), k+m-1) 



*iffnnm*<3ffCTfllffl£26} 



WO 95/19683 



PCT/USy5/00S63 



51 

By way of explanation, the sum of the or terms in equation 
[41] corresponds to the boundary wavelets for those points 
in image block 51 that are near the edges, and the sum of 
the q terms in equation [41] corresponds to the inner 
wavelets for those points in image block 51 that are away 
from the edges. 

According to the preferred embodiment of this 
invention, each of the matrices c", P*" 1 and Q^" 1 are pre- 
calculated and stored in program data memory 35 in each 
channel compression subsystem 29, in a manner accessible by 
digital matrix processor 30, so that the matrix operations 
62, 64 of Figure 10 can be readily and rapidly performed. 
The Appendix to this specification specifies examples of 
these matrices C**, P 1 *" 1 and Q N " 1 which have actually been 
used for decomposition of an image JblocJc ot a size 256 
pixels square, and where the numeric representation of 
matrices C, P and Q in the Appendix, are given for the cubic 
(m=4) case. 

Upon obtaining the intermediate decomposition 
coefficients in process 64, the final decomposition 
coefficient matrices c 11 " 1 and d 1 *" 1 , for the low-pass 
(spline) and band-pass (wavelet) components, respectively, 
are obtained by processes 66, 68, respectively. According 
to this preferred embodiment of the invention which 
utilizes matrix algebra, process 66 obtains the final low- 
pass decomposition coefficient matrix c**" 1 from the 
relationship : 

?? 

where the matrix is the inner product of the cardinal 

spline function 0(x), calculated according to equation [35] 
noted above for the next matrix in sequence. The values of 
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the matrix cN-1 , . . , ^, - 

correspond to the values m the low- 
frequency portion of processed image block 53 of Figure 9b, 

and thus represent the function f„ (x,y) in the spatial 

N-l 

domain. Matrix c therefore also represents the 
5 frequency domain representation f^-1 shown in Figure 8. 

Similarly, process 68 of Figure 10 determines the 
final decomposition coefficient matrix d N-1 for the band- 
pass component of the image block 51. Process 68 is based 
10 upon the operation: 

b 11 " 1 = dN-V" 1 [43] 

where matrix a known matrix that may be 

15 precalculated according to the relationship: 

D"- 1 = (Q N )V +1 (Q N ) [44] 

and is thus based upon known relationships. Upon 
20 completion of process 68, the matrix d**' 1 corresponds to 
the high-frequency values in processed image block 53 of 
Figure 9b, The matrix d N-1 thus also represents the 
spatial function Sjj-i^y)' and * ts corresponding frequency 
domain representation g^, which is the band-pass component 
25 shown in Figure 8. 

Upon completion of the horizontal decomposition of 

N-l N-l 

process 50, matrices c and d are stored in memory 
banks 36 of channel decomposition subsystem 29. preferably 

30 in the order illustrated in Figure 9b as processed image 
block 53. Referring back to Figure 7, process 52 is then 
performed to again decompose processed image block 53 in 
similar manner as process 50, only in a vertical direction, 
i.e., for each vertical column of processed image block 53, 

35 taken column-by- column. In other words, the decomposition 
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of process 52 is performed considering the image data for 
each vertical column of processed image block 53 as a one- 
dimensional spatial function in the y-dimension. According 
to the preferred embodiment of the invention, the matrix 
algebra used in process 52 is identical to that shown in 
Figures 10 and 11, for the next level of matrices in 
sequence, except on a column-by-column basis for each 
vertical column in processed image block 53 . 

Process 52 thus decomposes the low- frequency spatial 
function f N-1 (x,y) into a low- frequency portion "LL" and a 
higher- frequency portion "LH" contained within processed 
image block 55 as shown in Figure 9c, where the second 
decomposition is performed on processed image block 53 in 
a vertical direction. Figure 8 illustrates, in the 
frequency domain, that the matrix portion LL corresponds to 
the function ^-2(^-1) that is the low-pass portion of the - 
prior low-pass function f^-i, and that the matrix portion 
HL corresponds to the function g^-if^-i that is the band- 
pass portion of the prior low-pass function fjj-1. 

Similarly, process 52 decomposes the prior band-pass 
decomposition function g^-l into a low-frequency (i.e., 
lower band-pass) portion ("HL" in Figure 9c J and a high- 
frequency portion ("HH» in Figure 9c) , but where the second 
decomposition is performed on processed image block 53 in 
a vertical direction. Referring to Figure 8, this 
decomposition provides frequency domain representation 
f N' 2 %" 1 corresponding to the HL portion of final processed 
image block 55, and frequency domain representation %-2g N -l 
corresponding to the HH portion of final processed image 
block 55. 

The LL portion of final processed image block 55 
corresponds to the "blur" image of the input video image 
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block 51, and the LH, HL and HH portions of final processed 
image block 55 correspond to band-pass portions of the 
input video image block 51, taken in both the horizontal 
and vertical directions, successively. As is evident from 
5 the frequency domain representations of Figure 8, the LH, 
HL and HH components correspond to the components of the 
image in increasing frequency bands, with the HH component 
corresponding to the highest frequency component. 

10 The decomposition performed in processes 50, 52 

described hereinabove provide important benefits in the 
field of video image decomposition. Firstly, the benefits 
of time -frequency localization of the 1 signal in a manner in 
which the frequency-width of the window widens and the 

15 time-width of the window shrinks with increasing frequency 
provided by wavelet decomposition are obtained. This 
allows for more detailed and thorough analysis of rapidly 
changing portions of the signal, and thus a more accurate 
decomposition of the video image. Secondly, by using the 

20 boundary- spline -wavelet approach described hereinabove, 
boundary effects at the edges of the image or of subdivided 
image blocks within the image are eliminated. The 
elimination of these boundary effects allows for smaller 
capacity computers to perform the decomposition with no 

25 degradation in image quality. Elimination of boundary 
effects also enables each subdivided image to be processed 
independently by a stand-alone device or processor system, 
so that the entire decomposition may be performed by 
parallel processing or parallel computing, without 

30 extensive modifications. In addition, the decomposed image 
may, upon decompression, be magnified without the presence 
of boundary effect artifacts in the displayed image. 
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Numerica l decomposition example 

The decomposition of processes 50, 52 can be 
illustrated by way of a simple numerical example. In this 
example, the input image block is eight -by- eight pixels in 
size, and contains an eight -bit digital value 
representative of the display intensity at that location 
(i.e., is in PGM format). An example of the input image 
block in this format is as follows: 
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Process 50, as noted above, performs a boundary- 
spline -wavelet decomposition of the input image block, 
taken for each horizontal row. m this example, where a 
trivial zero degree (m-0) wavelet is used, for purposes of 
explanation, the horizontal decomposition of process 50 
results in the matrix: 
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The portion of the above matrix on the left-hand side of 
the divider corresponds to the low- frequency portion of the 
input image, while the portion of the above matrix on the 
right-hand side of the divider corresponds to the high- 
frequency portion of the input image, both performed in a 
horizontal manner. 



Process 52 then decomposes the result of process 50, 
considering the data in vertical columns as one- dimensional 
spatial sampled functions. Again using the same trivial 
zero-degree wavelet, the vertical decomposition of process 
52 provides the result: 
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The orientation of the above matrix uses the nomenclature 
of Figure 9c, as follows: 
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Video i mage decomposition examp le 

Referring now to Figures 12a through 12e, an example 
of the boundary-spline-wavelet decomposition according. to 
the preferred embodiment of the invention as applied to an 
actual video image will be described. Figure 12a 
illustrates an input video image in PGM format to which 
boundary- spline-wavelet decomposition is applied. Figures 
12b through 12e illustrate the LL, LH, HL and HH components 
of the final processed video image after performing 
processes 50, 52 of Figure 7, using the matrix operations 
described hereinabove relative to Figures 10 and 11. As is 
evident from Figure 12b, the LL component is a quite 
faithful representation of the input image of Figure 12a. 
In the LH, HL, HH components illustrated in Figures 12c 
through I2e, respectively, the visible striations 
correspond to higher frequency components o± the input 
image . 



b. Threshol ding and compression 

Referring back to Figure 6, upon the completion of the 
horizontal and vertical decomposition of the input image 
block performed in process 40, process 42 is next performed 
by quantization processor 32 upon the results of the 
decomposition stored in memory banks 36. 

Specifically, the LH, HL and HH components of, the 
decomposition are subjected to thresholding and 
quantization in process 42, with no such quantization 
performed upon component LL. This selection of the LH, HL 
and HH components for quantization is based upon the 
observation that most real-world images in PGM format will 
consist of low-frequency intensities, and that the higher- 
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frequency wavelet portions of the decomposition will tend 
to have a large number of small or zero coefficients. The 
numerical example set forth above relative to the 
decomposition of the eight -by- eight image illustrates this 
effect. In addition, the exemplary image of Figures 12a 
through 12e also shows that the LL component of the 
decomposition contains the most information, with the LH, 
HL and HH components containing only a small amount of 
significant information indicated by the striations 
therein. 

According to the preferred embodiment of the 
invention, therefore, this large population of small values 
in the higher- frequency components after decomposition may 
be either discarded (or reduced in memory requirements) by 
the quantization of process 42. For example, with 
reference to the above numerical example, all coefficients 
having an absolute value less than 2.0 may be set to zero 
(thresholding) , and all remaining coefficients may be 
rounded to their nearest signed integer value. The memory 
requirements for storage of the coefficients that undergo 
the thresholding and quantization are thus much reduced, 
even before the application of lossless compression 
techniques as will be noted below. 

Other thresholding and quantization techniques may 
alternatively be applied to the decomposed image data, 
selectable according to the code stored in quantization 
data register 31 described hereinabove. 
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Video image quant ization example 

Referring now to Figures 13a through 13c, an 
illustration of the result of thresholding performed- in 
process 42 is illustrated, relative to the video image 
decompression. example of Figures 12a through I2e. Figure 
13a is the same image as the LL -blur" component also shown 
in Figure 12b. Figure 13b illustrates, for example, the 
sum of the LH, HL and HH coefficients after the 
decomposition of process 40 of Figure 7, and thus is the 
sum of the images of Figures 12c through 12e discussed 
above. Figure 13c illustrates the sum image of Figure 13b 
after the application of a threshold limit and quantization 
of process 42. As is evident from Figure 13c, the higher 
frequency components LH, HL, HH from the decomposition of 
process 40 correspond only to the edges and other sharp 
transition locations of the input image. 



£j Completion at video cnmproaai™ 

Referring back to Figure 6, after the thresholding and 
quantization of the LH, HL, HH components of the decomposed 
input image performed in process 42, decision 43 determines 
whether the desired compression ratio has yet been reached. 



As illustrated particularly by the video and numerical 
examples noted above, upon the completion of a single 
decomposition process 40, the LL component of the image may 
be adequate to accurately convey the input image. This 
single decomposition of the image will provide up to a 4:1 
compression ratio, depending upon the memory requirements 
for the quantized high frequency results. This maximum 
ratio considers that the wholesale discarding of the higher 
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frequency components will leave only the LL component 
residing in a matrix that is one-fourth 'the size of the 
input image block. Specifically, therefore, decision 43 
determines if the compression ratio achieved so far is 
5 adequate for the desired transmission or storage and, if 
not, passes control back to the decomposition process so 
that the LL component resulting from the prior 
decomposition may again be decomposed into four more 
components . 

10 

According to this embodiment of the invention, the 
determination of whether the desired compression ratio has 
been obtained may be done relative to a predetermined 
compression ratio. In such a case, timing circuit 37 will 
15 maintain a count of the number of passes through 
decomposition process 40, and will perform decision 43 by 
comparing the resulting compression ratio against a 
previously stored value. 

20 Alternatively, the compression ratio and thus decision 

43 may be determined in a dynamic manner, frame by frame, 
depending upon the accuracy with which the LL component is 
representative of the input frame image. Conceptually, 
such a determination will be a measure of the difference 

25 between the LL component and the input image block relative 
to a predetermined accuracy limit, such that if the LL 
decomposition is within a preselected e limit, an 
additional pass through the decomposition process 40 may be 
performed. It is contemplated that this determination can 

30 be made automatically by main controller 28 in decompressor 
16 so that the process performed in each of channel 
compression subsystems may remain consistent, by 
calculating a numerical value based upon the sum of the 
coefficients in the LH, HL, and HH components of the 

35 decomposed image, such sum indicating the difference 
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between the input image to that pass of the decomposition 
process and the resultant LL component. 

in the usual case, where each video image frame is 
subdivided into smaller image blocks for compression, it is 
preferred that the compression ratio be constant for all 
compressed image blocks of the frame. Accordingly 
decision 43 is preferably performed after each image block 
has been decomposed and quantized in processes 40 ; 42 to 
allow the dynamic determination of whether to repeat 'the 
decomposition process 40 is made based on the worst case 
decomposed image. 

According to this dynamic determination of decision 
43, video image frames that do not contain high frequency 
information to any large extent, such as background images 
with smooth transitions between colors, may be compressed 
to a significantly higher ratio than can video image frames 
that do contain high-frequency information, or a large 
number of sharp transitions. Use of such dynamic 
compression can optimize the overall compression efficiency 
with minimal impact on the image quality. 

Whether statically or dynamically determined, upon 
decision 43 returning the result that the desired 
compression ratio has been obtained, according to the 
preferred embodiment of the invention, lossless compression 
is then performed upon the results ot the decomposed and 
quantized images for the frame, in process 44 of Figure 6 
Referring back to Figure 5, the lossless compression of 
process 42 is preferably performed by lossless compressor 
34 according to a conventional lossless technique such as 
Huffman encoding. The lossless compression of process 42 
is especially beneficial in compressing the quantized 
higher frequency LH, HL, HH components from the 
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decomposition, considering that non-zero or varying values 
in these components will be quite sparse for most video 
image frames. 

After the lossless compression of process 42, the 
compressed video image data is formatted, or coded, on a 
frame -by- frame basis for transmission or storage, as the 
case may be. As noted above, it is contemplated that most 
video image frames will be compressed according to the 
preferred embodiment of the invention after subdivision 
into multiple image blocks, such that a complete trame will 
consist of multiple ones of such blocks, transmitted 
together. As illustrated in Figure 4a, it is preferred 
that the coding of the compressed image data be performed 
within compressor 16, preferably by data flow interface 39 
shown, in Figure 5, prior to its application to digital 
communications network 15. 

Referring now to Figure 14, a schematic illustration 
of formatted frame 70 of data according to the preferred 
embodiment of the invention will now be described. As will 
be described in further detail hereinbelow, the 
decompression of video image data compressed according to 
the present invention can occur very rapidly; for example, 
portions of motion pictures compressed according to the 
present invention have been decompressed and displayed on 
a real-time basis. This rapid and accurate decompression 
allows for the ability of enhanced features to be used in 
the display of the decompressed images. The construction 
of data frame 70 of Figure 14 includes the compressed bit 
stream data corresponding to a video frame, plus the 
appropriate formatting "header" information to allow the 
enhanced features to be operable. The header portion of 
frame 70 preferably provides complete information to 
describe the frame and its position. 
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As illustrated in Figure 14, frame 70 is a sequential 
block of data formatted for a storage device such as 
computer memory, disk storage, CD-ROM and the like it is 
therefore contemplated that, in order to take advantage of 
all of the features of the present invention, compressed 
data frames 70 will be stored in a computer memory prior to 
its decompression, rather than decompressed in a real-time 
manner as received over digital network 15. Real-time 
decompression and display may alternatively be performed as 
the data is received, but certain ones of the features 
described hereinbelow will not be as useful in such a case 
Since the extraction of the header information from frame 
70 requires extremely little computing time and effort 
inclusion of this header information will have 
substantially no penalty in the overall performance of the 
real time decompression and display. 

The first portion of frame 70 is HI header 71, which 
xs data of string type used to specify the status of the 
bit stream compressed image, and as such may contain on the 
order of twenty bytes of data. For example, HI header 71 
may include an identifier of the movie of which the frame 
is a part, and may also contain a security code for 
preventing unauthorized viewing or use of the compressed 
data. For example, if the compressed video data 
corresponds to a movie being communicated over digital 
telephone lines, HI header may contain a code corresponding 
to an identifier of a local decompression system 20 so that 
only that decompression system will be enabled to 
decompress the data. Field 72 of frame 70 is a four^byte 
field of long integer type which contains the number of its 
corresponding frame 70 to specify the position of the frame 
20 in the entire video image sequence. Field 73 is a two- 
byte field of integer type which identifies the group of 
motion (if any) to which frame 70 belongs. a group of 
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frames 70 may be designated as a group, such that the 
frames in the group cannot be separated from one another or 
cut from the entire sequence. Such grouping can prevent 
unexpected side effects in the display of the motion 
picture sequence. 

Fields 74 and 7S then follow in frame 70 according to 
this example, to facilitate control of the display of the 
video sequence containing frame 70. Field 74 is a four- 
byte field of long integer type which contains the address 
at which the previous frame in the sequence begins, 
enabling rapid jumping back to the previous frame as 
desired. As will be described hereinbelow, a user control 
interface may be provided with decompressor system 20 to 
allow interactive control of the display of the video 
sequence, in which case field 74 will facilitate the 
skipping and selection of individual frames in reverse 
order. Similarly, field 75 is a four-byte field of long 
integer type which contains the address of the next frame 
in the sequence, allowing rapid skipping of frames 70 in 
the forward direction during decompression and display. 

Field 76 is a two-byte field of integer type that 
indicates the complexity of the image contained within 
frame 70, by specification of compression ratio, quality 
index, or a user-defined specification of the image, such 
values useful in measuring and controlling the performance 
of the decompression and display. 

Fields 77, 79, 81, 83 indicate certain parameters used 
in the compression of the video data for frame 70. The use 
of header information to communicate these parameters allow 
for decompression system 20 to have selectable 
capabilities, such that it can decompress video image data 
that were compressed according to different techniques, m 
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addition, since each frame 70 includes these fields 77 79 
81, 83, the provision of these fields according to the 
preferred embodiment of the invention enables dynamic 
compression, such that different compression techniques may 
be used on different frames 70 in the same video sequence. 
Field 77 is a two-byte field of integer type that specifies 
the level of decomposition in the compressed data of frame 
70, so that decompression system 20 may perform the proper 
levels of decompression upon receipt. Field 79 is an 
integer field indicating the type of lossless compression 
performed by lossless compressor 34, for example- 



0 
1 
2 



no lossless compression 
Huffman coding 
adaptive Huffman coding 



and so on. Ple ld 81 is a two-byte field of integer type 
that indicates the quantization mode used by quantization 
processor 32 described hereinabove relative to compression 
system 10, for example following the tabular listing of the 
codes noted above. Field 83 is a two-byte field of integer 
type that stores the quantization value used by 
quantization processor 32, and thus depends upon the mode 
indicated in field 81. Using the tabular example noted 
hereinabove for control of quantization processor 32, the 
value of field 83 will indicate the valug ag 

xollows : 
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Mode # (fjp 1d fl1) Field aa ~ r >- 



0 

1 

2 
3 



0 (don't care) 
thresholding value 
dequantizing scalar 
address of the JPEG 
quantization table, 
as an offset from 
the compressed data 
4 address of vector 

quantization table, 
as an offset from 
the compressed data 

As indicated in Figure 14, portion 78 of frame 70 
contains the compressed image data for field 70 and 
follows fields 87, 89. Field 87 is a four-byte fie'ld of 
long integer type that specifies the length of portion 78 
containing the compressed data stream, and is useful in 
assisting the reading of the compressed data stream, such 
as by way of a DMA operation. Field 89 is a four-byte 
field of long integer type that indicates the offset 
distance between field 87 and the start of portion 78 
containing the compressed image data; field 89 may thus be 
used to reserve space within frame 70 for other fields and 
information defined by the user or other developers. 

Portion 78 contains a data stream of the compressed 
video image data for frame 70, as noted above. The length 
of portion 78 will, of course, depend upon the compression 
ratio obtained in the compression process, upon the number 
of pixels in the frame, and also upon the extent to which 
the lossless compression of process 44 compressed the 
higher frequency components of the frame. In addition it 
is contemplated that audio information may also be encoded 
m a conventional manner, and placed within portion 78 of 
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llT"- SUPP ° rt transmis =i- « storage of sound 

motion pictures. 



Ttas arrangement of frame 70 is particularly useful in 

v^eoT aCtlVe deCO " PreSsi ° n — ««Play of a sequence' of 
Video frames. Specifically, fields 74. 75 and 7 6 enable 
decompressor 20 to flexibly display the frames in thT 

fralT^; eSPeCiallX ^ ^ Wh « e "» of 

' ™ r Se,Uential fr — = -"on picture. P or 
decompressor 20 can interrogate field 76 to 

its 7" ^ Pr ° CeSSi ^ «"P-ity of decompressor 20 and 
its display system 2S is such that every frame in the 
seance cannot be decompressed and displayed in real time; 
if so, decompressor 20 can skip to the next frame 70 in the 
seance seated by the contents of field ,s in frame 70 

aSld ^ T " P ° SSible iBa9eS * en ««— 
«Wd. those frames that are not skipped are displayed in 

real-txme, so that the time -dependence of the motion L t£ 
■notion picture is accurately conveyed. 

caoabil"^ ?4 ' 76 alS ° PrOVidS tive display 

capability. as field 74 indicates the address of thl 
previous frame in the sequence. frame ,o allows t^ 
capability of backwards display of a motion picture 

fY"*.**- 70 *" «» ««— or with frames 
pipped as described above based on the time required for 
decompression and the capability of decompressor „. In 
addition, the information provided by field 75 facilitates 
the synchronization of the display of the sequence of 
frames 70, and also allows for easy scaling of the tile 
base to provide slow-motion or enhanced-speed display 

Accordingly. referring back to Figure 6 , che 
formatting or coding of process 46 thus prepares the 
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compressed video image data for transmission over digital 
network 15 for storage, decompression and display by 
decompression system 20, or alternatively to disk unit 22 
for archival storage. 



Construrf ion of f he dsrnm r r af5 p ff r 

^ — Hioh performan ce decompressor 

Referring now to Figure 15a, the construction of 
decompressor 18 according to the preferred embodiment of 
the invention will now be described in detail. it is, of 
course, contemplated that other architectural arrangements 
of circuitry may be used to decompress video data according 
to the present invention. Specifically, it is contemplated 
that a conventional general purpose computer system may be 
capable of performing the compression of the video image 
data according to the present invention. However, the 
example of Figure 15a incorporates a preferred embodiment 
of the circuitry for performing the data decompression 
functions described herein. 

In this example, decompressor 18 preferably includes 
a data flow interface 80 coupled to network 15 to receive 
the incoming bitstream data of the compressed video image, 
as described above relative to Figure 14. Data flow 
interface 80 provides an interface to network 15, and 
serves to separate the R, G, and B components of the 
incoming bitstream into three channels, namely the R- 
stream, G-stream and B-stream channels. Decompressor 18 
also includes main controller 84 and three substantially 
identical channel decompressor subsystems 88 R , 88 , 88 
Main controller 84 preferably controls the operation 'of the 
functional circuitry within decompressor 18, and as such is 
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connected by way of the appropriate bidirectional buses and 
control lines to subsystems 88 and data flow interface 80 
to control the timing, feedback and receiving operation of 
decompressor 18. Main controller 84 is preferably a 
general purpose programmable microprocessor or other 
central processing unit (CPU) of sufficient computational 
power and capacity to process some or all of the image data 
and to control the performing of the image compression 
functions to be described hereinbelow. it is contemplated 
that microprocessors having performance levels similar to 
or greater than those of the 80486 type (including those 
available from Intel Corporation or Cyrix Corporation) , of 
the 68040 type (including those available from Motorola), 
and of the SPARC processor type (available from Texas 
Instruments Incorporated or Sun Microsystems, Inc.) will be 
suitable for use as main controller 84 in decompressor 18. 

The three channel compressor subsystems 88 R , 88 , 88 
each receive a corresponding one of the separated channels 
presented by data flow interface 80, so that the boundary- 
spline-wavelet decompression of the three color components 
is performed in parallel according to this embodiment of 
the invention. The construction of channel decompressor 
subsystems 86 is substantially the reverse of that of 
channel compressor subsystems 29 described hereinabove. 
Each of subsystems 88 R , 8 8(3 , 88 B is therefore constructed 
substantially identical to perform these similar tasks as 
the others, in parallel; for purposes of clarity, the 
following description is provided only for channel 
decompressor subsystem 88 R , it being understood that the 
construction and operation of the other subsystems 88 will 
be similar, if not identical, to subsystem 88_ described 
herein. 
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Each channel decompressor subsystem 88 according to 
this embodiment of the invention is specifically designed 
to perform the functions of lossless decomoression, 
dequantization, and boundary spline wavelet re -compos it ion 
(or decompression) , as used in the decompression operation 
according to this embodiment of the invention. Channel 
decompressor subsystem 88 according to this embodiment of 
the invention includes the circuit functions of digital 
matrix processor 86, timing circuit 107, dequantization 
processor 108, lossless decompressor 82, program data 
memory 112 , and multiple memory banks 110 tor storage of 
image data. 

The incoming data received by subsystem 88, as shown 
in Figure 15a, is received by lossless decompressor 82. 
Lossless compressor 82 may be implemented oy way ot a 
conventional digital signal processor such as the TMSC40 
programmed in such a manner as to perform lossless 
decompression upon the incoming data stream for that 
channel as presented by data flow interface 80. The 
lossless decompression performed by lossless decompressor 
82 may be according to a conventional technique, such as 
Huffman encoding, adaptive Huffman encoding, arithmetic 
encoding, LSQ coding, and the like; specifically, however, 
lossless decompressor 80 must either recognize (from field 
79 of frame 70) or be previously programmed to operate 
according to the lossless compression technique used in 
compression of the incoming data. Lossless decompressor 82 
stores the decompressed incoming image data in memory banks 
110, in preparation for the re -composition of the image 
data. Alternatively, lossless decompressor 82 may be 
implemented as a custom logic circuit for providing this 
function. 
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The output from lossless decompressor 82 corresponds 
to the decomposed image data for that channel (R, g, B) , in 
identical form as that presented to lossless compressor 34 
in compressor IS, as described hereinabove. Dequantization 
processor 108 is a logic circuit or programmable device 
such as a microprocessor, for de-quantize the output from 
lossless decompressor 82. According to this preferred 
embodiment of the invention, dequantization processor 108 
is controllable to operate according to various 
quantization or thresholding modes, according to the 
contents of dequantization code register 109 which receives 
the contents of field 81 of frame 70. The dequantization 
value in field 83 is used in the dequantization performed 
by dequantization processor 108. The results of the 
dequantization are stored in memory banks 110. 

Digital matrix processor 86 performs the boundary- 
spline -wavelet composition upon the results of the 
dequantization, . using pre -calculated matrices stored in 
program data 112 . The procedure for this composition is 
described in detail hereinbelow. The results of the 
operations performed by digital matrix processor 86 are 
stored in memory banks 110. As in the case of compressor 
16, digital matrix processor 86 may be implemented by way 
of a conventional digital signal processor, such as the 
i860 microprocessor available from Intel Corporation and 
the TMSC40 digital signal processor available from Texas 
Instruments Incorporated, or by way of a general purpose 
microprocessor such as those of the 80386 and 80486 
families available from Intel Corporation, and of the 68030 
and 68040 families available from Motorola. 

Timing circuit 107 performs the functions of 
controlling the storage of the channel data from lossless 
decompressor 82, and the subsequent retrieval thereof for 
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dequantization and decompression processes. Upon 
completion of the decompression of the channel data for a 
frame, timing circuit 107 is operable to present the 
decompressed PGM channel data to main controller 84, for 
formatting and transmission to format converter 24 for 
display at display 26. 

Main controller 84 receives the decompressed channel 
data from each of subsystems 88 R , 88 G , 88 B and sequences 
the data into the format suitable for receipt by format 
converter 24. In addition, main controller 84 operates to 
provide feedback to data flow interface 80, such that upon 
decompression of a frame of data, data flow interface 80 
can be controlled to communicate the next frame of data to 
subsystems 88 for decompression. 

In addition, main controller 84 is coupled to user 
interface device 85, which may be a mouse, trackball, pen 
input,- keyboard or the like; in addition, user interface 
device 85 may be implemented as another computer, such as 
a personal computer workstation, through which a user 
controls the decompression and display of video data. As 
noted hereinabove, the construction of frame 70 shown in 
Figure 14 facilitates the interactive control of the 
decompression process, such that the user may control the 
direction (forward/backward) and rate of decompression and 
display of the sequence of video image frames, using user 
interface device 85. Main controller 84 is thus operable 
to communicate to data flow interface 80 the order, and 
selection of frames for decompression; it is therefore 
useful to incorporate adequate memory for the storage of 
multiple frames either within or accessible by data flow 
interface 80 to allow such interactive decompression and 
display. 
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the ell ^pressor l 8 ls intended to support 

the concession of high definition real- time true ZlZ 
video lma3e data, where -true color" indicates the use of 

results ln l 6 . 7 mlnion ^ 
dec 0mp ion race for decompressor ^ ^ ^ 

implementation, color data could * " 

sequentially for the R G L T decompressed 
con , rnl . . ' G and B components under the 

control of main controller 84 t« m **>*. 

perform the lossless decompression, as well. 

The construction of decompressor i 8 d^^K „ 

"^1 " hi ~ ~ -™ 

a the Mlth ^ hi ^-anc. workstation eu h 
as the 4D series workstations available fr™, <=■,. 
graphics, and SPARC workstations ^J^Z lZ 

aae"TT' M th6Se »— PerforJceTevet 

adequate for delivering „ore than thirty fran.es per selond 

ZJt :°:T ti0 * <80 ° PlXelS * 600 ^> ^ 9 e ^ a 
with 24-bit true color capability. 
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^ ^sona! -compter ^ ard „ n 

system 

Referring now to Figure 19, decompression system 200 
according to an alternative embodiment of the invention 
will now be described in detail. Decompression system 200 
is mtended for implementation on a conventional PC-based 
workstation, and is configured as a single 8-bit expansion 
board; alternatively, system 200 may be configured as a 16- 
bit ISA or EISA expansion board, or as a MCA (Micro Channel 
Architecture) board, m each case, it is contemplated that 
decompression system 200 will be suitable for medium 
resolution displays. 

Decompression system 200 includes lossless 
decompressor 202, which is implemented as a video 
processing board. Lossless decompressor 202 has an input 
port coupled to network 15 (or, of course, to disk storage 
or another source of video image data compressed according 
to the method described hereinabove). Lossless 
decompressor 202 includes digital signal processor (DSP) 
204 whxch is a conventional digital signal processor such 
as the TMSC25 and TMSC30 types available from Texas 
instruments Incorporated, or a general purpose 

TrZTT e T r 8UCh M 68020 ° r 68030 ^ e liable 

from Motorola. Program memory 206 is preferably EPROM or 

other conventional firmware storage, for storing the 
programs utilized by DSP 204 in performing the lossless 
decompression upon the received data. 

Lossless decompressor 202 receives the compressed RGB- 
PGM data from, network is and performs the necessary 
lossless decompression according to the lossless 
compression technique used (if any) in compressor 16 it 
is preferred that lossless decompressor 202 be capable of 
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performing decompression of various types, selectable by 
way of field 79 in frame 70, described hereinabove relative 
to Figure 14. Lossless decompressor 202 is also coupled to 
user interface device 205, to allow a human user the 
capability of controlling motion control in the 
decompression and display of a sequence of video images. 
Lossless decompressor 202 presents its output to 
decompression processor 210 for the boundary- spline -wavelet 
decompression . 



Decompression processor 210 according to this 
embodiment of the invention includes processing capability 
to perform boundary-spline-wavelet decomposition according 
to the preferred embodiment of the invention described 
herein. The main controller of decompression processor 210 
is digital signal processor (DSP) 225, which is preferably 
a high-performance single-chip digital signal processor, 
such as the TMSC40 available from Texas Instruments 
Incorporated or the i860 processor available from Intel 
Corporation, or a high-performance general purpose 
microprocessor such as the 80486 microprocessor available 
from Intel Corporation. Decompression processor 210 also 
includes program data memory 212 for storage of program 
code and pre -calculated matrices useful in the 
decompression process, data memory banks 214 for storage of 
image data, and digital matrix processor 220 for performing 
matrix operations useful in the decompression routine. 
Digital matrix processor 220 is preferably a conventional 
digital signal processor, such as the i860 microprocessor 
available from Intel Corporation and the TMSC40 digital 
signal processor available from Texas Instruments 
Incorporated, or a general purpose microprocessor such as 
those of the 80386 and 80486 families available from Intel 
Corporation, and of the 68030 and 68040 families available 
from Motorola. Host interface controller 208 is also 
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provided within decompression processor 210 for 
communicating with a host computer (not shown) including 
the video display associated therewith. Program memory 222 
is also provided in decompression processor 210 and 
contains the program code for boundary spline wavelet 
reconstruction and format conversion. Video timing control 
circuit 217 is a i so contained within decompression 
processor 210, for controlling the viewing of the 
decompressed data as well as the timing of the 
decompression, so that a sequence of video frames may be 
vxewed in real-time as a movie, or under the control of 
user interface device 205. 

In operation, lossless decompressor 202 stores the 
results of the lossless decompression for a frame in memory 
banks 214. These data are retrieved by DSP 225 via memory 
interface 224, according to a sequence stored in program 
memory 222, and forwarded to digital matrix processor 220 
along with precalculated matrices stored in program data 
212 along the coefficient bus shown in Figure 19. Digital 
matrix processor 220 performs matrix operations upon the 
image data presented by DSP 225 to perform the boundary- 
splme-wavelet decomposition and forwards the results to 
DSP 225 along the data-out bus shown in Figure 19 for 
storage in memory banks 214. Digital matrix processor 220 
also provides control information to DSP 225 on the 
feedback bus therebetween. 



Upon completion of the boundary- spline -wavelet 
reconstruction or decompression, the decompressed image 
data for a frame is stored in memory banks 214. Memory 
interface 224 enables the interleaved or simultaneous 
accessing of memory banks 214 for boundary-spline-wavelet 
decompression and display; host interface controller 208 
accesses memory banks 214 for display purposes. Host 
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10 



15 



20 



25 



30 



35 



interface controller 208 may also have sufficient 
capability to directly display the contents of memory banks 
214 containing the decompressed video image data- 
alternatively, the decompressed image data may be forwarded 
by host interface controller 208 to memory of the host PC 
for eventual display, by way of DMA. 

It is contemplated, of course, that still further 
alternatives to the decompression hardware may be utilized 
according to the present invention. 

S, Pounds-spline.^ pf „ ld » n imafTm 

and dl«m» Y 

Referring now to Figure 16, a method according to the 
preferred embodiment of the invention, for decompressing 
video xmage data that was compressed by way of boundary- 
spline-wavelet compression as discussed above, will now be 
described m detail. This decompression will be described 
relative to the hardware decompression. system of Figures 4a 

wf J 53 ,; ^ bSing UnderSt °° d that analogous operations 
would be carried out by decompression system 200 
illustrated in Figure i 9 , for its particular 
implementation . 



The initial step of the decompression method of this 
example is the reading of the compressed file or sequence 
of frames 70 by data flow interface 80, indicated in Figure 
16 by process 90. Process 90 may be the result of the 
receipt of communicated compressed video image data by data 
flow interface 80 directly from digital network 15 
Alternatively, where decompressor is is a portion of a 
stand-alone computer, disk unit 22 or another fixed storage 
unit may contain the file of compressed video image frames 



mum iiuur MHtrmn r ie\ 



WO 95/19683 



PCT/US95/005d3 



'8 

" " In 



from an earlier transmission, or as archival storage m 
either case, the result of process 90 is the receipt of a 
sequence of compressed video image frames ,o by data flow 

Figure 16 can retrieve individual frames 70. 

Process 91 is next performed. where each of the 
channels of a selected frame 70 n undergoes lossless 

Compressed in^a^ e^rd^T /he Preferred 
embedment of the invention, and as such the process fT™ 

a si:r e h s f T this *** fotwara " U1 be ^ 

» "ngle channel <E, G, B) of frame ,<,. The particular 

"Z ne iS t r l8Cted aCC ° rdln3 E ° «*— <* -me. 
the user v PiCtU "' " al "™"vely selected by 

the user via user interface device 85. In either case 

To so™" 84 em ' the ~— ° f 

™„. so that the appropriate frame 70 is presented to 

Channel decompressor subsystems ... which provTde the d at a 

for frame 7 0 n to lossless decompressor ... noted ahove 

the lossless decompression performed in process 91 bv 

loss es decompressor 82 corresponds Vthe type of 

lossless compression performed by lossless compressor^* ll 

compressor «. ,he result of the lossless degression of 
proces, ^ provides ^ finai deco ^ sed £ 3ion of 

relative »' " 

relative to the compression method. 

After the lossless decompression of process 91, the 
image data is deouantized for each channel R, o, B % 
deouantization processor l„s in its corresponding channel 

P=rforme e d 8 b "d SUbSySten ° f ^-"za^n 

performed by denization processor 10. is controlled by 

the contents of denization data register 109, whij 



WO 95/19683 



PCT/US95/00563 



79 



contains a value corresponding to that transmitted as field 
81 in frame 70 n . In the event that the data was quantized, 
the quantization value transmitted in field 83 of frame 70 
is utilized to restore . the data to that prior to 
quantization in compressor 16. 

Upon completion of the dequantization of process 92, 
process 93 is performed for each image block of frame 70 ' 
to reconstruct the full video image in PGM format, with 
each pixel location containing a value corresponding to the 
intensity at that location. In effect, process 93 reverses 
the decomposition process described hereinabove relative to 
Figures 6 and 7, according to the boundary- spline -wavelet 
approach. This reconstruction of process 93 is performed 
on a block-by-block basis, for each of the image blocks in 
the selected frame 70 n . Upon completion of process 93, the 
selected frame 70 n is stored as a PGM format frame in 
memory banks 110. 



Referring now to Figure 17, a method for 
reconstructing the selected frame 70 Q according to the 
preferred embodiment of the invention will now be described 
in further detail. Recalling that the decomposition of the 
video image data performed during compression is performed 
first in the horizontal direction and then in the vertical 
direction, the decomposition must be performed in the 
reverse order. The first step in the reconstruction of the 
video image data is therefore the reconstruction of each 
column in the vertical direction, as indicated by process 
98. 

The reconstruction process -98 is also most 
efficiently, and thus preferably, performed by digital 
matrix processor 86 in the corresponding channel 
decompression subsystem 88 using matrix algebra operations 
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similarly as i„ the egression process described 
hereinabove. Accordingly, process 9 a begins by consider^ 
the image block in columns, each containing a set of 

* b ina "he""/ "* frS ™ P °" ion ' 

If 3 nUB,ber ° f tiMS th « »°undary-spline-wavelet 
decomposition was performed during concession to obtaL 

olressi =0mPreSSi ° n ra "°- FM ™- «— ^ 
expression ratl o was 4=!. the value of k would eoual a 

once)': r!r / ' deC °"*° si "°* — Perform 

once), for a compression ratio of 64=1, k would equal e: 

used ST " U1UStrateS tne -l^ionship of the matrices 
Zees" 3 "instruction of the image block performed in 
directed to ^^^^ ^ration (process ,., is 

values ! 3 ^ ^ ^ ° rder ™«icient 

N-k+1 

ma ^. ; Conversel y to the case of compression, the 

matnx operation is equivalent to: 



c N-k + l = pN -k c N-k + Q N-k d N-k 



[45] 



»h«e the matrices P* and oF are the spline and wavelet 
-trices respectively, for level N, as used in the 
decomposition and compression process. As not ed Lve 
matrices P « and gP can be pre-calculated and stored itThe 
Program data memory « of channel decompression subsystl 

Process 9a thus performs one level of the matri* 
operati o£ equation M} ^ , co 

each column in the image block, tfcon completion of P ™ 
«. Process „. performs a similar reconstruction as tlat 
in process ... only on a row-by-ro„ basis for each row^n 
the image block. As such, the operation of equation^ 
Performed to obtain the coefficients c^ 1 " 2 . 
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It will, of course, be noted that the decompression of 
the video image data according to the preferred embodiment 
of the invention will not result in an exact duplicate of 
the input image presented . to the compression process, as 
the effects of the quantization and thresholding process in 
the compression of the video image data cannot be full 
recovered (i.e., some data is lost). As such, the 
boundary-spline-wavelet compression according to the 
present invention is a lossy compression technique, it is 
contemplated, however, that the use of the boundary- spline - 
wavelet approach described hereinabove causes the loss of 
only that data at high-frequencies, such that the loss of 
quality of the image is minimized for a given compression 
ratio. 



However, it should especially be noted that the 
decompression processes 98, 100 can be done quite rapidly 
particularly with specialized circuitry such as a digital 
matrix processor performing the matrix operations noted 
above. This speed in processing arises from the wavelet 
forms selected in the compression, which correspond to 
express mathematical formulae and which are also quite 
adaptable to matrix operation techniques. As such, it is 
contemplated that the decompression performed by 
decompressor 18 can occur quite rapidly and, for many 
applications, need not require high performance 
microprocessor circuitry. 

Upon the reconstruction in both the column and row 
directions as performed in processes 98, 100, decision 101 
is performed to determine if the image block has been fully 
reconstructed (i.e., if the image block corresponds to a 
PGM format image, and does not contain low and high 
frequency components) . if not, the matrix operations of 
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processes 98, 100 are repeated to perform the next levels 
of reconstruction of the image for that image block. 

Upon process 100 completing the reconstruction of the 
image block, decision 103 tests whether more image blocks 
remain to be reconstructed in the selected frame 70 if 
so, the next image block is retrieved from memory 80 
(process 104) and is reconstructed by processes 98, loo 
101 as before. Upon completion of the reconstruction of 
all image blocks in selected frame 70 n , process 106 is 
performed so that the reconstructed image blocks in frame 
70 are stored in image buffer 88, arranged in the proper 
order to correspond to a full PGM format image frame 
completing process 93 of Figure 16 for a single frame 70 .' 
The decompressed image data for the frame 70 n , including 
that for all three channels R, G, B, are the forwarded t! 
main controller 84 for communication to format converter 
24. Main controller 84 then indicates completion of a 
frame to data flow interface- and performs decision 95 to 
determine if additional frames are to be decompressed and 
display; if so , main controller 84 will select the next 
frame for decomposition in the sequence, or as directed by 
user interface device 85. 

Referring back to Figure 16, process 94 is then 
performed by way of which format converter 24 reformats and 
communicates the frame to display 26, in the desired 
fashion. It is contemplated that format converter 24 may 
perform such operations as gathering three successive, PGM 
format frames, corresponding to the red, green, and blue 
color components of the same image, and presenting the data 
in RGB -PGM format to display 26; format converter 24 may 
also convert the RGB -PGM format data into another format 
such as PCX, IMG, GIF, TIF, RLE, ntsc and the ^ ^ 
appropriate for display 26. 
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Referring now to Figure 15b/ the construction of 
format converter 24 according to the preferred embodiment 
of the invention will now be described in detail. Format 
converter 24 includes three buffers 1 20r , iao c , 120 D for 
receding from decompressor IS and storing PGM format data 
corresponding to the three decompressed R, g, B channels 
The outputs of buffers 120 are received by color combiner 
Circuit 122, which is a data processing circuit such as a 
microprocessor, which combines the decompressed video data 
for tne three R, G, B channels into RGB-PGM format, color 
combiner 122 presents the results of this combination to 
the appropriate formatting circuitry for driving display 
26 m the analog case, color combiner 122 presents its 
• output to video digital-to-analog converter (DAC) 126- 
vxdeo DAC 126 is a conventional video DAC suitable for 
receiving digital video or graphics information and driving 
ana! g di lay 26a . ^ ^ ^ ^ ^ ^ng 

122 preeents lts output to color reducer ^ forraafc ^ 

124 Reducer/encoder 124 is a conventional circuit for 
encoding the RGB -PGM format data presented thereto by color 
combiner 12 2 into the desired format for digital display 
26d, such formats including TIF, IMG, GIF and the like Of 

rTpIm if /T ay ^ 13 CaFablS ° f di —^ ^splaying 
RGB -PGM video data, reducer/encoder 124 is unnecessary 

ar S J 11 additi ° n ' f0rmat inverter 24 may also include 
graphics processing capability so as to perform more 
graphics processing operations, including magnification 
rooming and the li ke . A s noted above, the elimination of 
boundary effects resulting from the boundary- spline -wavelet 
compression and decompression according to the present 
invention is especially beneficial when such complex 
graphics processing features. For example, a portion of 
the image of frame 70 n may be readily magnified according 
to conventional graphics processing techniques, such as the 
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use of interpolation techniques to fill in pixels between 
those for which the frame contains actual data, with a high 
degree of accuracy. m contrast, boundary effects that 
result from prior compression techniques tend to be 
exaggerated when the image is magnified, considering not 
only the magnified display but also the exaggeration 
resulting from interpolation between "true" image data and 
the boundary effect artifacts. 

Indeed, it is contemplated that the present invention 
wxll be especially beneficial for the transmission and 
storage of compressed video images for display on advanced 
dxsplays that have higher pixel densities than the density 
of the image so compressed, due to the elimination of 
boundary effect artifacts by the present invention. m 
add-on, it is contemplated that such magnification will 
allow for the display of the images on a plurality of video 
displays arranged in an array, with each display showing 
only a portion of the full image. 

Referring back to Figure 16, upon transmission of the 
selected frame 7<> n to format converter 24 performed in 
process 94, decision 95 is then performed to determine if 
additional frames in the sequence are to be decompressed 
for display, if SO/ process 96 selects the next frame 70 
for decompression and display. According to the preferred 
embodiment of the invention, the selection of process 96 
may be made in multiple ways. a first approach for 
selecting the next frame for decompression and display is 
merely to select the next frame in the sequence, and access 
the same by way of field 75 (Figure 14) from the previously 
decompressed frame 70^ this approach will be followed 
during the decompression and display of a motion picture 
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As noted above relative to Figure 14, field 76 

ratiTL l indiCa " nS SUCh £a=t0rS M "» ^ssion 

comoLT tha * indiC " e the ^"ational 

complexity required for the decompressor, and display of 

next 7" all °" S £ ° r intelli *=" "iection of the 

next frame in process 96, wher e the capability of 
decompression system 20 is limited relative to the amount 
of compressed video data in the transmission. For example, 
if the time that decompressor 18 win require for 
decompression and display and display of a frame is longer 
than the reciprocal of the frequency at which frames are to 
be displayed (e.g., longer than !/30 sec. in the case of a 
conventional motion picture) , main controller 84 will skip 

decor' " e " framSS ' Md SeU « * *« 

^ » «-t the sequence of frames 

that are displayed will appear in real-rim. * >.• 
display 26. real-time fashion on 



in J „ f ° r tha Bele « i! » °« the next frame 

in process 96 is that which may be directed by the user via 
user interface device 65. For example, the viewer may wish 
or i« y ' 3 basis JZt 

backed direction, or the viewer may wish to repetitively 
view a eelected portion of the sequence. These commands 
are conveyed to main controller 84 by user interface device 
85, and are used in process 96 according to the present 
invention to select the next frame. . 

Opon the decompression and display of all of the 
frames in the sequence as determined by decision 95, the 
process of decompression and display ends 
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5 . Conclusinn 



The methods and systems for compressing and 
decompressing video image data described hereinabove 
relatxve to the present invention provide important 
advantages, as noted throughout the • foregoing 
speculation. These advantages include the fundamental 
benefits of wavelet analysis, where time- frequency 
localization of the input signal is implemented so that the 
t lme window narrows with increasing frequency and widens 
with decreasing frequency, thus providing highly accurate 
analysis for transient periods of the signal, and avoiding 
the undersampling problems of conventional Fourier analysis 
techniques. 

In addition, the video i mage compression and 

eLoTT° n teChniqU6S aCCOrdin3 t0 the Purred 
embodiments of the invention utilize boundary- spline- 
wavelets to eliminate boundary effects in the compressed 
images, specifically by using different wavelets for 
samples near the boundaries of an interval from those 
wavelets used for inner samples. The boundary wavelets do 
not require support from outside of the interval; when 
applied to video compression according to the present 
invention, therefore, boundary effect artifacts are 
eliminated. This not only provides an extremely accurate 
representation of the original input image but, when 
coupled with the computational geometrical superiority of 
spline functions, also enables the performance of complex 
processing operations and display such as magnification 
dynamic compression on a frame-by-frame basis, interactive 
dxsplay of a motion image, including insertion, editing and 
repetitive display. j» addition, the present invention 
also enables slow display systems to skip frames so that a 
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motion picture can be displayed on a ~i - ■ u 

fewer F-^yea on a real-time basis with 

tewer trames per second. 

The elimination of boundary effects also enables th e 
5 compression routine to be i ■ ena *>J-es the 

parallel processing techniques to be readily applied to til 

while still oh,, T impression and decompression. 



15 



20 



25 



30 



35 



in addition, the present invention also enable „ 
Ration pr ovlded by ~ ZLZZLZZSZZZ 

enaoj.es the dynamic compressinn 

™-=^ 

The present invention may be amU^i - 

applications of video ima ^ ° Vari ° US 

viaeo xmage compression, such » = 

transmission of motion pictures and other vide! 
between remote locations, such a ^ 
c = cation network or by satellite transTssion 
present invention may also be app lie d advantageously to T 
archival storage of video image data, both TtTonXur 
and stm ln ,aae data, in a compressed for*. The archivll 

acilitated by the present invention due to the lack J 
Boundary effects, as the decompression and display of tne 



WO 95/19683 



PCT/US95/00563 



88 



compressed information may be performed in much higher 
density and capability systems than those used in the 
original compression, without exaggeration of artifacts 
such as boundary effects. 

While the invention has been described herein relative 
to its preferred embodiments, it is of course contemplated 
that modifications of, and alternatives to, these 
embodiments, such modifications and alternatives obtaining 
the advantages and benefits of. this invention, will be 
apparent to those of ordinary skill in the art having 
reference to this specification and its drawings. it is 
contemplated that such modifications and alternatives are 
within the scope of this invention as subsequently claimed 
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WE CLAIM: 

1. A method of compressing digital data 

representative of a video image for communication thereof, 
5 comprising the steps of: 

formatting the digital data corresponding to a 
frame of the video image into an array of locations in rows 
and columns, with the value at each array location 
corresponding to the intensity of the display; 
10 decomposing, in a first direction, each row of 

said array into low-frequency and high-frequency portions 
using boundary-spline-wavelets, and arranging the results 
of the decomposing into corresponding rows; 

then decomposing each column of said arranged 
15 results of the first decomposing step in a second 
direction, using boundary-spline-wavelets; and 

communicating the results of the decomposing 
steps to a receiving unit. 

20 2. The method of claim 1, wherein said communicating 

step comprises: 

storing the results of the decomposing steps in 
a fixed memory. 

25 3. The method of claim l, wherein said communicating 

step comprises: 

transmitting the results of the decomposing steps 
to a decompression system for display. 

30 4. The method of claim 3, further comprising: 

after receiving the transmitted results of the 
decomposing steps, reconstructing the transmitted video 
image in the second direction, using boundary-spline- 
wavelets; 
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then reconstructing the results „ f -k ^. 
reconstructing step i„ the f irst di * 
spline-wavelets; and ^ bo ^ry- 

- steps on :illl7Z£: ° f ^ —cting 

10 freguency <»J^ Z^'*™"* 0 ^ t0 l0W " 

first decomposing step; direction of the 

frequency eo J^ ' =™o M1 „ g to the iow- 

"igh-freguency deco^ion in^T* dlreCtl0 "' ° f 
» ««t decomposing .^LT ' """"" °* «» 

freguency comp^eT t^l 0 T: h COrreSPOndin9 <° 

first decomposing step; Erection of the 

25 and further comprising; 

.. The *w , of elila „ f ^ 

». HL, and HH components —•«—»« on the 

7. The method of claim 5, further comprising; 
repeating said decomposing steps. 
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8. The method of claim 5, further comprising: 

transmitting the LL component and the quantized 
LH, HL, HH components over a communications network; 

receiving the transmitted components from the 
5 communications network; 

reconstructing a low-frequency component from the 
LL and quantized LH components using boundary-spline- 
wavelets in the second direction, and reconstructing a 
high-frequency component from the quantized HL and HH 
10 components using boundary-spline-wavelets in the second 
direction; 

then reconstructing a video image from said low- 
frequency component and said high-frequency component using 

boundary-spline-wavelets applied in the first direction; 
15 and 

displaying the results of the reconstructing 
steps on a video display. 

9. The method of claim 8, further comprising: 
20 after said quantizing step and prior to said 

communicating step, performing lossless compression on the 
LL, LH, HL, and HH components; and 

after said receiving step, performing lossless 
decompression on the communicated LL, LH, H L and HH 
25 components. 

10. The method of claim 8, further comprising: 
repeating said decomposing steps a selected 

number of times prior to said communicating step; and 
30 repeating said reconstructing steps the selected 

number of times prior to said displaying step. 

11. The method of claim 5, further comprising: 
prior to said decomposing steps, dividing each 

35 frame into a plurality of image blocks; 



WO 95/19683 



PCT/US95/00563 



94 



10 



wherein said decomposing steps are performed for each 
image block of each frame, so that each of said image 
blocks comprises a ll component and quantized LH, hl, and 
HH components. 

12. The method of claim ll, further comprising: 

transmitting the LL component and the quantized 
LH f HL, HH components for. each of said image blocks of each 
frame over a communications network; 

receiving the transmitted components from the 
communications network; 

for each of said image blocks, reconstructing a 
low-frequency component from the LL and quantized LH 
components using boundary-spline-wavelets in the second 
direction, and reconstructing a high-frequency component 
from the quantized HL and HH components using boundary- 
splme-wavelets in the second direction; 

then, for each of said image blocks, 
reconstructing a video image from said low-frequency 
20 component and said high-frequency component using boundary- 
spline-wavelets applied in the first direction; 

arranging the reconstructed video image for each 
of said image blocks into a video frame; and 

displaying the video frame on a video display. 

25 

13. A method of displaying compressed video image 
data, comprising: 

receiving compressed video image data 
corresponding to a frame and storing the compressed frame 
30 ln memory, said compressed video image data arranged as 
first, second, third and fourth frequency domain 
components ; 



in a 



first image direction, applying boundary- 
spline-wavelets to each column of data in the compressed 
35 frame to reconstruct a low-frequency component from the 
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first and second frequency domain components, and to 
reconstruct a high-frequency component from the third and 
fourth frequency domain components; in a second 

image direction, applying boundary-spline-wavelets to each 
row of data of the low-frequency component and high- ' 
frequency component to reconstruct the video image frame- 

displaying the reconstructed video image frame on 

a vadeo display. 

10 14. The method of claim 13, further comprising: 

repeating said steps of applying boundary-spline- 



- — i-jr— j — »<g */wuuwiaiy— spin 

Z^T! " first Md second toa " 

selected number of times. 



a 



■rn.iv- . aeth ° d ° f Claim 13 ' Wherein said ^eps of 
applying boundary-spline-wavelets are performed, for each 
frame, for a plurality of image blocks of said frame. 

16. The method of claim 13, further comprising: 
performing lossless decompression on the 

compressed video image data, prior to said applying steps! 

17. The method of claim 13, further comprising- 

25 r~ «. aft6r applyin * ste P s * storing the 

25 reconstructed video image frame in an image buffer. 

18. The method of claim 17, further comprising: 
after said storing step, magnifying the image of 



30 



said video image frame prior to said displaying step." 

\«* ^ I- A SySten f ° r comnunic ^ing video image 
information, comprising: 

an input source for providing digital video image 
information arranged as frames; and 
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compressor circuitry having an input coupled to 
said input source, for decomposing each frame of digital 
video image information in a first image direction using a 
boundary-spline-wavelet into a first low-frequency 
5 component and a first high-frequency component, and for 
then further decomposing each of said low-frequency and 
high-frequency components of each frame of digital video 
image information in a second image direction using a 
boundary-spline-wavelet into first and second pairs of low- 
10 frequency and second high-frequency components, said first 
pair being the decomposed representation of said first low- 
frequency component and said second pair corresponding to 
a decomposed representation of said first high-frequency 
component, said compressor circuitry also having an output 
15 for presenting the first and second pairs of low-frequency 
and high-frequency components. 

20. The system of claim 19, further comprising: 

a format converter circuit coupled between said 
20 input video source and said compressor, for converting the 
digital input video information into a portable gray-level 
format . 

21. The system of claim 20, wherein said input source 
25 provides color video information; 

and wherein said format converter circuit is for 
converting the color video information into a plurality of 
portable gray-level format representations, each 
representation corresponding to a color component. 

30 

22. The system of claim 19, wherein said compressor 
circuitry comprises: 

decomposing circuitry for performing the 
decomposing of each frame of digital video image 
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information into the fircf = _ j « , 

tile iirst and second pairs of low- 
frequency and high-frequency components; 

quantization circuitry for quantizing the high- 
frequency component of said first pair, and for quantizing 
5 the both the low-frequency and the high-frequency component 
of the second pair; and 

lossless compression circuitry, having an input 
coupled to said quantization circuitry, for performing 

10 IZ l S \ C T T&SSim ° f ^ ^ S6COnd ^"-frequency 

10 and high-frequency pairs, prior to presenting the 

decomposed frames of digital video image information at the 
output of the compressor circuitry. 

_ 23. The system of claim 22, further comprising: 

mam controller circuitry, for controlling the 
operation of the decomposing circuitry, so that the low- 
frequency component of said first pair may be repetitively 
provided to the decomposing circuitry for further 
decomposing. 

20 

24. The system of claim 19, wherein said compressor 
circuitry comprises: 

an image buffer coupled to the input of the 

25 ClrCUitry f ° r St ° rin * a ***** ot digital video 

25 image information; and 

a digital matrix processor for performing the 
decomposing of each frame of digital video image 
information into the first and second pairs of iow- 
frequency and . high-frequency components using a boundary- 
30 spline-wavelet. 

25. The system of claim 24, wherein said digital 
matrix processor operates according to matrix operations 
using precalculated matrices corresponding to spline and 
35 wavelet components and stored in said compressor circuitry 
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26. The system of claim 19, further comprising: 

a storage unit, coupled to the output of said 
compressor circuitry, for storing the decomposed frames of 
digital video image information. 

27. The system of claim 19, wherein the output of 
said compressor circuitry is coupled to a digital network 
for communication of said decomposed frames of digital 
video image information. 



28. The system of claim 27, wherein said compressor 
circuitry comprises: 

decomposing circuitry for performing the 
decomposing of each frame of digital video image 
15 information into the first and second pairs of low- 
frequency and high-frequency components; 

quantization circuitry for quantizing the high- 
frequency component of said first pair, and for quantizing 
the both the low-frequency and the high-frequency component 
20 of the second pair; and 

lossless compression circuitry, having an input 
coupled to said quantization circuitry, for performing 
lossless compression of the first and second -low- frequency 
and high-frequency pairs, prior to presenting the 
25 decomposed frames of digital video image information at the 
output of the compressor circuitry; 

and wherein the output of the lossless compression 
circuitry is coupled to the digital network so that the 
30 decomposed frames of digital video image information are 
communicated after the lossless compression. 

29. The system of claim 19, further comprising: 

a communications network coupled to the output of 
35 the compressor circuitry; 
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decompressor circuitry, having an input coupled 
to said communications network, for reconstructing each 
decomposed frame of digital video image information in the 
second image direction using a boundary-spline-wavelet, and 
5 for then further reconstructing each frame of decomposed 
digital video image information in the first image 
direction using a boundary-spline-wavelet; and 

a video display, for displaying the reconstructed 
frames of digital video image information. 

10 

30. The system of claim 29, wherein said compressor 
circuitry comprises: 

decomposing circuitry for performing the 
decomposing of each frame of digital video image 
15 information into the first and second pairs of low- 
frequency and high-frequency components; 

quantization circuitry for quantizing the high- 
frequency component of said first pair, and for quantizing 
the both the low-frequency and the high-frequency component 
20 of the second pair; and 

and wherein said decompressor circuitry comprises: 

reconstructing circuitry, for reconstructing a 
digital output video frame from the decomposed frames of 
digital video image information, by reconstructing an 
25 approximation of the first low-frequency component from the 
first low-frequency and high-frequency pair in the second 
image direction, and an approximation of the first high- 
frequency component from the second low-frequency and high- 
frequency pair in the second image direction, and by then 
30 reconstructing the digital output video frame from the 
approximations of the first low-frequency component and the 
first high-frequency component in the first image 
direction. 
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31. The system of claim 30, wherein the compressor 
circuitry further comprises: 

main controller circuitry for controlling the 
operation of the decomposing circuitry, so that the low- 
5 frequency component of said first pair may be repetitively 
provided to the decomposing circuitry for further 
decomposing. 

and wherein the decompressor circuitry further 
comprises : 

10 main controller circuitry, for controlling the 

, operation of the reconstructing circuitry so that the 
result of the reconstructing may be repetitively operated 
upon by the reconstructing circuitry, depending upon the 
number of times the decomposing circuitry repetitively 

15 decomposed the digital input video frame. 

32. The system of claim 30, wherein said compressor 
circuitry further comprises: 

lossless compression circuitry, having an input 

20 coupled to said quantization circuitry, for performing 
lossless compression of the first and second low-frequency 
and high-frequency pairs, prior to presenting the 
decomposed frames of digital video image information to the 
communications network; 

25 and wherein said decompressor circuitry further 

comprises: 

lossless decompression circuitry, having an input 
coupled to the communications network for performing 
lossless decompression of the digital video image 
30 information received therefrom. 

33. A system for displaying compressed video image 
data , comprising : 

a memory for storing a plurality of compressed 
35 video frames; 
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decompressor circuitry for reconstructing each of 
said plurality of compressed frames in a first image 
direction using a boundary-spline-wavelet, and for then 
further reconstructing each of said plurality of compressed 
5 frames in a second image direction using a boundary-spline- 
wavelet; and 

a video display , coupled to said decompressor 
circuitry, for displaying each of said plurality of 
reconstructed frames. 

10 

34. The system of claim 33, wherein said memory is 
coupled to a communications network. 

35. The system of claim 33, wherein said decompressor 
15 circuitry comprises: 

a digital matrix processor coupled to said 
memory, for reconstructing each of said plurality of frames 
using precalculated matrices. 

20 36 • The system of claim 33, wherein each of said 

plurality of frames are stored in said memory in a form of 
the type comprising quantized first and second pairs of 
low- frequency and high-frequency components, said first 
pair of low-frequency and high-frequency components 

25 corresponding to the decomposition, in the first direction, 
of a low-frequency decomposition in the second direction of 
a video image frame, and said second pair of low- frequency 
and high-frequency components corresponding to the 
decomposition, in tfhe first direction, of a high-frequency 

30 decomposition in the second direction of the video image 
frame. 

37. The system of claim 36, wherein each of said 
plurality of frames are stored in said memory in a form 
35 including the lossless compression of said quantized first 
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and second pairs of low-frequency and high-frequency 
components ; 

and further comprising: 

lossless decompression circuitry, coupled between 
5 said memory and said decompressor circuitry, for performing 
lossless decompression of each of said plurality of frames 
prior to the reconstruction thereof by said decompressor 
circuitry, 

10 38. The system of claim 36, wherein said first and 

second pairs of low-frequency and high-frequency components 
correspond to the results of repetitively decomposed 
components of a video image; 
and further comprising: 

15 main controller circuitry, for controlling the 

operation of the decompressing circuitry so that the result 
of the reconstructing may be repetitively operated upon by 
the decompressing circuitry according to the number of 
times that the stored frames were repetitively decomposed. 

20 

39. The system of claim 33, wherein each of said 
plurality of frames stored in said memory correspond to a 
plurality of compressed image blocks; 

and wherein said decompressor circuitry reconstructs 

25 each of the compressed image blocks for each of said 
plurality of compressed frames in the first image direction 
using a boundary-spline-wavelet, and for then further 
reconstructing each of the compressed image blocks for said 
plurality of compressed frames in the second image 

30 direction using a boundary-spline-wavelet. 
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EP 0 622 741 A2 



Timet 


0 •••• 3 


4 7 


8 •••• 11 


Input Data 
Values 


HHHqo • • • HHG(j-j 


HHH 10 --HHGv 


HHH 2 o • • HHG 2 1 1 


muxsel(1) 


1 •••• 1 


1 1 


1 •••• 1 I 


muxsei(2j 


2 2 


1 1 


1 •••• 1 | 


muxsel(3) 


3 •••• 3 


2 2 


2 2 1 


andsel(1),(4) 


pass " ** pass 


zero — zero 


pass — pass 1 


andsel(2),(3) 


zero — zero 


pass — pass 


pass * * * * pass 1 


addsel(1) 


add — add 


add ** ** add 


add — add ' 


addsei(2) 


add — add 


sub — sub 


add • • • • add 1 


addsel(3) 


add ** ** add 


add • • • • add 


add * ■ ** add ■ 


addsei(4) 


sub — sub 


add — add 


sub — sub ! 


centermuxelM) 

WW 1 1 »W ■ 1 1 1 W\W II 1 I 


1 — 1 


> . • . . f 


1 — l 

■ 1 1 


centermuxel(2) 


r • ■ • * r 


, 


r . . . . r 

11 


muxandsel(l) 


zero — zero 


)ass *■ * * pass 


pass — pass 1 


muxandse!(2) 


pass ■ * # • pass 


pass — pass 


pass — pass 


muxandsel(3) 


zero — zero 


Dass — pass 


pass • • • ■ pass 


OUT2 
OUT1 




4 
\ 


2HHHH 0 o--32HHGH 0 il 


OUTPUT 
LEADS 524 


FIG. 


19A 


HHHHqo • • HHGHqi 1 



CONV_COL CONTROL SIGNALS AND OUTPUTS 
DURING THE FORWARD OCTAVE 1 TRANSFORM 
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12 



1 



zero 



pass 



add 



sub 



add 



add 



pass 



pass 



15 



HHH30 * • HHG31 



1 



zero 



pass 



add 



sub 



add 



add 



pass — pass 



pass 



pass 



2HHHGQQ. • 32HHGG 0 1 



HHHG00..HHGG01 



16 



pass 



zero 



add 



add 



add 



sub 



zero 



pass 



zero 



19 



pass 



zero 



add 



add 



add 



sub 



zero 



pass 



zero 



32HHHH 10 »32HHGH 11 



HHHH 10 ..HHGH 11 



20 



zero 



pass 



add 



sub 



add 



add 



pass • 



pass * 



pass 



23 

1 
1 
2 

zero 

pass 

add 

sub 

add 

add 



pass 
pass 
pass 



32HHHG 10 .. 32HHGGii 



FIG. 19B 



HHHG10 .. HHGG-| 1 
KEY TO FIG 19 



CONV_COL CONTROL SIGNALS AND OUTPUTS 
DURING THE FORWARD OCTAVE 1 TRANSFORM 



FIG. 


FIG. 


19A 


19B 



FIG 19 
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EP 0 622 741 A2 

Input 
Data 



t 


Value 


Output of Block 1326 


—I 


0 


HHHqo 




1 


• 






— 1 

1 


3 


HHGqi 






4 


HHH 10 


32aHHH 10 




• 
• 




• 


1 

— 1 


7 


HHGn 


32aHHG-|i 


1 


8 


HHH 20 


32HHHGoo = 32{(c+d)HHH 0 n - bHHH 10 + aHHHon} 

WW » * ' WW 1 W fav' 




■ 








A 4 
11 


HHQ2-I 


32HHGGq^ = 32{(c+d)HHGoi - bHHGii + aHHG2i) 


' 


12 


HHH30 












J 


15 


HHG 31 






16 




32HHHG 10 = 32{dHHH 10 +CHHH20 - (b-a)HHH3oJ 




• 
• 




• 


| 


19 




32HHGG 11 =32{dHHGn +HHG21 -(b-a)HHG 31 } 













FIG. 20A 

CONV.COL DATA FLOW FOR FORWARD 
OCTAVE 1 TRANSFORM 
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EP 0 622 741 A2 



| Output of Block 1328 


Output of Block 1330 


i 32(a+b)HHH 0 o 


32(c+d)HHH 0 o 


1 • 

l ♦ 
■ | 




I 32(a+b)HHG 0 i 


32(c+d)HHG 0 i 


32{(c+d)HHH 00 -bHHH 10 } 


32{(a+b)HHHoo + cHHH 10 } I 


j ; 


I 


32{(c+d)HHG 0 i-bHHG 11 } 


32{(a+b)HHG 0 i+cHHG 11 } | 


32{aHHH 10 +bHHH 2 o} 


32{dHHH 10 +cHHH 20 } 


• 
• 




32{aHHGn +bHHG 21 } 


32{dHHG 11 +cHHG 2 i) 


| 32{dHHH 10 + cHHH20-(b-a)HHH3o} 


32{aHHH 10 + bHHK^o + (c-d)HHH 30 } | 






]32{dHHG 11 + CHHG21 - (b-a)HHG 31 } 


32{aHHG 11 + bHHG 20 + (c-d)HHG 31 }J 


FIG. , 


20B 



CONV_COL DATA FLOW FOR FORWARD 
OCTAVE 1 TRANSFORM 
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Output of Block 1332 



32dHHH 10 



32dHHG 1 i 





32{(a+b)HHHoo + cHHH-i 0 - dHHH^ = 32HHHHqo 








32{(a+b)HHG 0 i + cHHGi 1 - dHHG 2 i} =32HHGHqi 















32{aHHH 10 +bHHH 2 o + (c-d)HHH 3 Q}= 32HHHH 10 



32{aHHG 1 ! + bHHG 2 i + (c-d)HHG 31 } = 32HHGH-j 1 



KEY TO FIG. 20 



FIG. 20C 

CONV.COL DATA FLOW FOR FORWARD 
OCTAVE 1 TRANSFORM 



FIG. 


FIG. 


FIG. 


20A 


20B 


20C 



FIG. 20 
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Time t 



muxsel(1) 2 



8 



11 



muxsel(2) 3 



muxsel(3) 1 



andse!(1),(4) zero zero 



pass 



pass 



zero 



zero 



andsel(2) zero — zero 



pass 



pass 



pass 



pass 



andsel(3) pass 



— pass 



pass — pass 



pass 



pass 



addsel(1) add 



add 



add 



add 



add 



add 



addsel(2) add 



add 



sub 



sub 



add 



add 



addsel(3) add 



add 



add 



add 



add 



add 



addsel(4) sub 



sub 



add 



add 



sub 



sub 



centermuxel(1)| r 



centermuxei(2) I 



muxandsel(l) pass • • • • pass 



pass 



pass 



pass — pass 



muxandsel(2) zero 



zero 



pass 



pass 



pass — pass 



muxandsel(3) pass 



pass 



zero 



zero 



pass 



pass 



OUT2 
OUT1 



8HHHoo--8HHGoi 



OUTPUT 
LEADS 524 



HHH 00 .. HHG01 



FIG. 22A 



CONV.COL CONTROL SIGNALS AND OUTPUTS 
FOR INVERSE OCTAVE 1 TRANSFORM 
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I 12 •••• 15 1 16 •••• 19 


20 • • • • 23 


I 2 2 2 — 2 


4 4 


'2 — 2 3 3 


1 1 


I 1 •••• 1 j1 •••• 1 


1 •••• 1 


| pass — pass zero — zero 


pass — pass 


J pass — pass J zero — zero 


pass * - * * pass 


1 pass — pass | pass — pass 


pass — pass 



add 



sub 



add 



add 



add 



add 



add 



sub 



add 



add 



add 



add 



add 



add 



sub 



• sub 



• • » • 



pass • • • • pass 



pass 



pass 



pass — pass I 2ero 



zero 



pass pass I pggg — p ass 



16HHHio-16HHGn 



16HHH 2(r 1€HHG 21 



HHH-jo • • • HHGn' HHH 20 • - ■ • HHG 2 1 



add 



sub 



add 



add 



pass 



zero 



add 



sub 



add 



- add 



• r 



pass — pass 



pass 



zero 



HHH30 • • 8HHG31 



HHH30 . . HHG 31 



FIG. 22B 

CONV_COL CONTROL SIGNALS AND OUTPUTS 
FOR INVERSE OCTAVE 1 TRANSFORM 



KEY TO FIG 22 



FIG. 


FIG. 


22A 


22B 



FIG 22 
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EP 0 622 741 A2 

OUTPUT OF BLOCK 1326 



32cHHHH 0 o 



3 |HHGH 01 



32cHHGH 0 i 



4 HHHGqo 8HHHoo=32{(b-a)HHHHoo+(c-d)HHHGoo} 



7 HHGGqiI 8HHG 0 i=32{(b-a)HHGH 01 +(c-d)HHGG 0 i} 



8 HHHH 10 



32(c+d)HHHH 10 



11 HHGHn 



32(c+d)HHGH 11 



12 |HHHG 10 | I6HHH20 = 32(-dHHHH 0 o +aHHHGQQ +bHHHH 10 +cHHGG 10 } 



15JHHGG-J ] [I6HHG21 = 32{-dHHGHoi +aHHGG 0 i +bHHGHu +CHHHG11} 



1 



16 

20 
23 



FIG. 23A 

CONVCOL DATA FLOW FOR INVERSE 
OCTAVE 1 TRANSFORM 
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EP 0 622 741 A2 



I OUTPUT OF BLOCK 1 328 


OUTPUT OF BLOCK 1 330 


i 32(b-a)HHHHoo 












32(b-a)HHGH 0 i 






32{cHHHHoo-bHHHGoo} 


32{-dHHHHoo + aHHHG 0 ()} 


i 




i 32{cHHGH 0 i-bHHGGoi} 


32{-dHHGH 0 i+aHHGGoi} ; 


32{-dHHHHoo + aHHHGoo +bHHHH 10 } 


32{cHHHHqo - bHHHGoo +aHHHH 10 }| 


- 


• 


32{-dHHGH 0 i + aHHGG 0 i +bHHGH 11 } 


32{cHHGH 0 i - bHHGGoi +aHHGH-|-j} 


32{(c+d)HHHH 10 -(a+b)HHHG 10 } 






• • 






j 32{(c+d)HHGH 1 1 -{a+b)HHGGii} 








32{(c+d)HHHH 10 -(a+b)HHHG 10 } 




32{(c+d)HHGH 11 -(a+b)HHGG 11 } j 











FIG. 23B 

CONV.COL DATA FLOW FOR INVERSE 
OCTAVE 1 TRANSFORM 
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EP 0 622 741 A2 



OUTPUT OF BLOCK 1332 



32{-dHHHH 00 } 



32(-dHHGH 01 } 



?{cHHHHoo-bHHHGoo + aHHHH 10 +dHHHG 10 )= 16HHH 10 



>{cHHGHqi - bHHGGoi + aHHGHoi +dHHGG 1l }= 16HHG^ 



32{(c+d)HHHH 1Q -(a+b)HHHG 10 } = 8 HHH 30 



32{(c+d)HHGHi 1 -(a+bJHHGGn) = 8 HHG 31 



FIG. 23C 



KEY TO FIG. 2c 




FIG. 23A 


FIG. 23B 


FIG.23C 



CONV.COL DATA FLOW FOR INVERSE 
OCTAVE 1 TRANSFORM 
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Timet 

INPUT DATA 
VALUE |HHHqo 



muxsel(1) I 2 
muxsel(2) 3 

muxsel(3) I 1 

andsel(1),(4) |zero 
andsel(2) zero 

andsel(3) IPass 



addsel(1) 
addsel(2) 
addsel(3) 
addse!(4) 



add 
add 
add 
sub 



centermuxsel (1) | r 
centermuxsei (2) 

muxandsel (1) Ipass 
muxandsel(2) zero 
muxandsel (3) pass 



OUT 2 
OUT1 



OUTPUT 
LEADS 520 



HHGq 



4 
1 
1 

pass 
pass 
pass 

add 
sub 
add 
add 

I 
r 

pass 
pass 
zero 



HHH 0 1 HHG 0 



3 
1 
1 

zero 
pass 
pass 

add 
add 
add 

sub 

r 
I 

pass 
pass 
pass 



8HH 00 



2 
2 
1 

pass 
pass 
pass 

add 
sub 
add 
add 

I 
r 

pass 
pass 
pass 



HH 00 



HHH-jo 



2 
3 
1 

zero 
zero 
pass 

add 
add 
add 
sub 



I 

pass 
zero 
pass 

16HH 0 i 
I6HH02 



HH01 



HH 02 



FIG. 24 

CONV.ROW CONTROL SIGNALS AND OUTPUTS 
FOR INVERSE OCTAVE 1 TRANSFORM 
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EP 0 622 741 A2 



Timet 


0 •■ 


•• 7 


8 ■ 


•■• 15 


16 • 


••• 23 


muxsel(1) 


2 •■ 


•• 2 


4 •■ 


* * 4 


2 • 


■••2 J 


muxsel(2) 


3 


■• 3 


1 •■ 


•• 1 


1 •■ 


" 1 


muxsel(3) 


1 


•• 1 


1 • 


•• 1 


1 •■ 


•• 1 I 


andsel(1),(4) 


zero •• 


•• zero 


pass - 


••• pass 


zero • 


••• zero | 


andsel(2) 


zero 


•• zero 


pass •• 


•• pass 


pass - • 


• • pass 


andsel(3) 


pass • - 


-' pass 


pass •• 


.. pass 


pass • • 


• • pass | 


addsel(l) 


add •■ 


•• add 


add 


•• add 


add •• 


•add 1 


addsel(2) 


add •• 


•• add 


sub •• 


•• sub 


add 


-add 1 


addsel(3) 


add 


add 


add •• 


add 


add 


■add • 


addsel(4) 


sub •• 


sub 


add 


•• add 


sub •• 


•sub 


wvl llwIIIIUAdl 1 J 


r 


r 






• • r 1 


centermuxel(2) 


1 •• 


■• 1 


r 


r 


1 


• • 1 

■ 1 


muxandsel(l) 


pass * * ■ 


• pass 


pass •• 


•• pass 


pass • • 


•• pass | 


muxandse!(2) 


zero • • ■ 


• zero 


pass • • ■ 


•• pass 


pass 


•• pass 1 


muxandsel(3) 


pass • - 


■ pass 


zero •• 


• zero 


pass • - 


pass 


out2 

0Ut1 






8H00-- 


• 8G03 | 


OUTPUT 






H00 ... 


. G03 « 



LEADS 524 

FIG. 26A 

CONV.COL CONTROL SIGNALS AND OUTPUTS 
DURING INVERSE OCTAVE 0 TRANSFORM 
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24 • 


••• 31 


32 


■ ■ • ■ 39 





48 • 


55 




2 • 


... 2 


2 


... 2 





3 • 


3 




1 • 


... -i 


1 


... -, 


.... 


1 • 


... -j I 




1 


... 1 


1 


... -j 





1 •• 


1 I 




pass ■ ■ 


pass 


zero •• 


•• zero 




zero • ■ 


■• zero | 




pass • - 


pass 


pass • 


... pass 





pass •• 


pass 1 




pass - • 


•• pass 


pass • 


• • • pass 





pass 


•• pass I 




add ■• 


•• add 


add • 


... add 





add •• 


•add ' 




sub 


.. sub 


add - 


• • • add 


■ • a * 


add •• 


••add I 




add •• 


.. add 


add - 


• • • add 


■ ■ • • 


add •• 


••add 1 




add •• 


. . add 


sub • 


• • • sub 




sub * > 


• • sub 1 




i 


i 

■ • i 


r 


■ • • l 




r •• 


•• r 1 




r 


. . r 


r 


... 1 


• • • • 


I 


•• 1 | 




pass • • 


. . pass 


□ass • ■ 




• . • • 


[J ado 






pass • • 


. ■ pass 


pass •• 


• * pass 


• • • • 


pass ■ ■ 


•- pass 




pass • • 


• • pass 


pass •• 


pass 




pass •• 


•• pass 




16H 10 


• 16G 13 


6H 20 ... 


* I6G23 




6H 40 ■• 


•• I6G43I 


t 


H10 -.. 


. G13 


H 2 0 


G 23 




H40 


• G 43 1 



FIG. 26B 

CONV_COL CONTROL SIGNALS AND OUTPUTS 
DURING INVERSE OCTAVE 0 TRANSFORM 
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56 



pass 



pass 



add 



sub 



add 



add 



pass 



pass 



63 64 



71 



pass 



zero 



zero 



pass 



zero 



zero 



pass — pass ) pass — pass 



• add 



add 



• • • • 



add 



sub 



add 



add 



add add 



add 



add 



sub 



sub 



pass — pass pass — pass 



pass 



zero 



zero 



pass 



pass — pass 



I6H50 I6G53 



16H 60 ---- 16G 63 



H 50 • • • • G53 I H 60 . . . . G 63 



72 



add 



sub 



add 



add 



zero 



H70 



79 



pass — pass 



pass — pass 



pass • • • ■ pass 



add 



sub 



add 



add 



* • • ■ 



pass • • • • pass 



pass — pass 



zero 



H70 8G73 



. G73 

KEY TO FIG 26 



FIG. 26C 

CONV.COL CONTROL SIGNALS AND OUTPUTS 
DURING INVERSE OCTAVE 0 TRANSFORM 



FIG. 


FIG. 


FIG. 


26A 


26B 


26C 



G 26 
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EP 0 622 741 A2 

Input 
Time Data 



t 


Value 


Output of Block 1326 




0 


HHqo 


32cHHqo 








• 




7 


GHno 


o^CunQ3 




8 


HGqo 

ww 


8Hoo= 32{(b-a)HHoo +(c-d)HGoo} 




• 




• 
• 




15 


GGo3 


0^03- *j<HiD-ajariQ3 +IC<jjciuq3j 




16 


HH 10 


32cHH 10 


— 


• 




• 






bM 13 


32cGHi3 




24 


HG 10 


I6H00 = 32{-dHHno +aHGnn +bHH 1 Q+cHG 10 } 

fc»W * WW WW 1 V 1 W' 




• 






j 


31 


GG13 


I6G23 = 32{-dGHo3 +aGG 03 +DGH13+CGG13} 




32 


HH 20 


32cHH 2u 




• 
• 








39 


GH 23 


32cGH23 








FIG. 27A 





CONV.COL DATA FLOW FOR THE 
INVERSE OCTAVE 0 TRANSFORM 
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EP 0 622 741 A2 



I Output of Block 1328 


Output of Block 1330 | 


. 32(b-a)HH 0 o 




1 


• 




: 1 

1 


I 32(b-a)GH03 






32cHHoo-32bHGoo 


-32dHHnn +■ 32aHGnn I 


1 


• 1 


i 32cGHq3 - 32bGGo3 


-32dGHo3 + 32aGGo3 | 


J 32{-dHHoo + aHGoo +bHH 1 o) 


32fcHHnn - bHGnn +aHI-h nl ' 


J 


• I 


32{-dGHo3 + aGGo3+bGH 13 } 


32{cGHo3-bGG 0 3+aGH 13 } | 


j 32cHH 10 -32bHG 10 


-acanri'ig + o^btmj^q i 


j 




J 32cGH 13 -32bGG 13 


-32dGH 13 + 32aGG 13 J 


32{-dHH 10 + aHGj 0 +bHH 2u } 


32{cHH 10 -bHG 10 +aHH 2 o} 






| 32{-dGHi 3 + aGGi3 +DGH23} 


32{cGH-| 3 - bGG-|3 +aGH 2 3} j 







FIG. 27B 

CONV.COL DATA FLOW FOR THE 
INVERSE OCTAVE 0 TRANSFORM 
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EP 0 622 741 A2 



J 


Output of Block 1330 




-32dHHoo 


j 






•32dGH<)3 








1 






j 






— 


-32dHHm 

\Jtm\Jl II I | II 




• 




-32dGHi 3 




32{cHHoo-bHGoo + aHH 10 +dHG 10 }=16H 10 








32{cGHo3 - bGGoa + aGH-j3 +dGGi3}= I6G13 




•32dHH20 




• 




-32dGH 2 3 







FIG. 27C 

C0NV.C0L DATA FLOW FOR THE 
INVERSE OCTAVE 0 TRANSFORM 
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EP 0 622 741 A2 











48 


HH30 


32(c+d)HH3o 












55 


GH33 


32(c+d)GH3o 




56 


HG 30 


16H 60 =32{-dHH 20 


+ aHG20 + bHH3Q + CHG30} 




• 










63 


GG33 


1 6Gg3= 32{-dGH23 + aGG23 + bGH33 + CGG33} 




64 










• 










71 










72 










• 










79 



















FIG. 27D 

CONV.COL DATA FLOW FOR THE 
INVERSE OCTAVE 0 TRANSFORM 
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EP 0 622 741 A2 





o^(Cnri20 • Dnu20 +3nn3Q/ • 


■ • 




j 32{-dGH 2 i + aGG2i +bGH 31 } 


32{cGH 2 i-bGG 2 i+aGH 31 } j 


| 32(c+d)HH3o-32(a+b)HG3o 








j 32(c+d)GH33 - 32(a+b)GG33 





32(c+d)HH3o - 32(a+b)HG3Q 
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